kotools 2026.7.14__tar.gz

kotools-2026.7.14/.github/workflows/publish.yml

@@ -0,0 +1,17 @@
+name: publish
+
+on:
+  push:
+    tags: ["v*"]
+
+jobs:
+  pypi:
+    runs-on: ubuntu-latest
+    environment: pypi
+    permissions:
+      id-token: write # OIDC — PyPI trusted publishing, no tokens
+    steps:
+      - uses: actions/checkout@v5
+      - uses: astral-sh/setup-uv@v7
+      - run: uv build --no-sources
+      - run: uv publish

kotools-2026.7.14/.gitignore

@@ -0,0 +1,21 @@
+__pycache__/
+*.py[cod]
+*.egg-info/
+.venv/
+.pytest_cache/
+.ruff_cache/
+dist/
+build/
+.DS_Store
+.claude/settings.local.json
+.arxiv2md_cache/
+
+# node tooling for `ko publish` (wrangler) — package.json + lock are committed
+node_modules/
+.wrangler/
+
+# never commit real credentials or tokens
+google_client.json
+google_token.json
+*.secret.*
+.env

kotools-2026.7.14/.python-version

@@ -0,0 +1 @@
+3.14

kotools-2026.7.14/AGENTS.md

@@ -0,0 +1,61 @@
+# AGENTS.md — driving `ko` from a shell
+
+`ko` is an opinionated CLI of thin SDK wrappers (web search, papers, HN, Google
+Sheets/Docs/Calendar/Gmail, X, publishing, LLM/agents). It's built to be driven by
+agents over bash as much as by a human. This file is the contract; `ko <cmd> --help`
+is the detailed contract for any one command.
+
+## First move
+
+Run `ko doctor` — it reports, per tool, what env/keys/binaries it needs and whether
+they're present. If a command fails with an auth/key error, `ko doctor` tells you why.
+
+## Output contract
+
+- **stdout is data; stderr is everything else.** Results go to stdout; errors, notes,
+  and "no results" messages go to stderr. You can pipe stdout into `jq`/`cut`/`awk`
+  without it being polluted by status text.
+- **Default format is plain text / TSV** — `cut -f N` safe. `ko gsheets get` and other
+  tabular commands emit TSV.
+- **`--json` for structured output.** Most read commands take `--json`. The shape is a
+  **JSON array of objects** unless the command's `--help` says otherwise (a few return a
+  single object — e.g. `ko tv --json` is `{top, matches}`, `ko hn item --json` is
+  `{story, comments}`, `ko papers get --json` is a single work object, `ko hf info --json`
+  is a single paper object — and `ko gsheets get --json` is a 2D array).
+  - Under `--json`, **errors are a JSON object on stderr**: `{"error": "...", "code": "..."}`.
+  - Under `--json`, **empty results are `[]` on stdout** (plus a note on stderr), so a
+    downstream `| jq` never chokes.
+
+## Exit codes
+
+- `0` — success, **including empty results** (empty is not an error).
+- `1` — runtime error (network, auth, bad upstream). One clean line on stderr; no traceback.
+- `2` — usage error (bad flags/args; Typer prints usage to stderr).
+
+## Limits / pagination
+
+- List commands take `--n` (default ~10) to bound output. Note: `ko gmail` / `ko cal`
+  currently use `-n` as the short alias; most other commands use `--n` with no short form.
+  When in doubt, check `ko <cmd> --help`.
+
+## Bare-argument shortcuts (not visible in `--help`)
+
+`ko` routes a leading bare argument deterministically (explicit command names always win):
+
+- `ko <url>` → `ko fetch <url>` (URL → markdown; YouTube URLs return the transcript, or fail cleanly when captions are off)
+- `ko <file>` → `ko doc <file>` (PDF/Office/image → text), when the arg is an existing file
+- `ko x <name>` → `ko x list <name>` (anything after `x` that isn't an x subcommand is a list name)
+- `ko <anything> help` → `ko <anything> --help` (trailing `help` prints help at any level: `ko help`, `ko exa help`, `ko exa search help`)
+
+## Auth
+
+Google commands (`gsheets`/`gdocs`/`cal`/`gmail`) share one OAuth token per account.
+`ko gsheets auth` grants it (covers Sheets+Docs+Calendar read+write, Gmail read-only).
+No browser? The only interactive prompt in the whole CLI is that one-time OAuth consent.
+
+## Working on the code
+
+- Python 3.14, `uv`, `ruff`, `pytest`, `typer`.
+- Install/run: `uv run ko ...`. Test: `uv run pytest -q`. Lint: `uv run ruff check`.
+- One module per domain under `src/ko/`; `cli.py` registers the Typer subapps. See
+  `CLAUDE.md` for the full design principles and how to add a subcommand.

kotools-2026.7.14/CHANGELOG.md

@@ -0,0 +1,106 @@
+# Changelog
+
+All notable changes to `ko` (kotools) are documented in this file. It is the
+canonical release record — GitHub release notes are drawn from it. WORKLOG.md is
+the dev journal (why/decisions); this is what changed, release-grained.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/2.0.0/).
+Versioning is [CalVer](https://calver.org/), PEP 440-normalized: `YYYY.M.D`
+(tag `vYYYY.M.D`; a second release the same day appends `.1`). The version covers
+the **CLI contract** (commands, flags, output shapes, exit codes — see AGENTS.md);
+breaking changes to it carry a `**Breaking:**` marker inline under their type.
+Cut a release with the repo's `/release` skill.
+
+## [Unreleased]
+
+## [2026.7.14] - 2026-07-14
+
+The money-and-agents release: model tiers with per-call cost notes, agents that
+remember between runs, and releases that publish themselves to PyPI.
+
+### Added
+
+- **PyPI publishing**: a tag push now builds and publishes `kotools` via GitHub
+  OIDC trusted publishing (`.github/workflows/publish.yml`, no tokens) — this
+  release is the first through the pipeline; `uv tool install kotools` from
+  here on.
+- `ko ai "<prompt>"` — the default agent: every toolset (web, papers, HN,
+  movies, read-only ~/code files) plus its own memory; medium-tier model,
+  hard-capped at 30 model requests per run. One-shot or REPL.
+- `ko agent repo` — repo-explorer agent: "how does repo X do Y" over `~/code`
+  (read-only by construction; ripgrep search; knows the refs/CLAUDE.md map;
+  cites file:line; basic tier — under a cent a run).
+- Agent memory: research + repo agents keep a per-agent markdown workspace
+  (`memory.md` anchor injected each run + free-form notes; append/edit tools
+  with a uniqueness guard), plus a shared hand-edited `~/.config/ko/memory.md`
+  injected into every memory-carrying agent (research, repo, ai). A fresh run
+  recalls what a prior run saved. Hardened after a trace-eval + review pass:
+  huge/minified files can't blow the context, broken symlinks and rg timeouts
+  degrade cleanly, and `grep` grew the `limit` param the model reached for.
+- Model tiers: `[llm] basic/medium/smart/ultra` in config.toml (`llm.model_for`) —
+  basic (deepseek-v4-flash) drives llm/tv/brief/summarize defaults, smart
+  (`~x-ai/grok-latest`) the research agent, ultra (gpt-5.6-sol) is reserved for
+  high-stakes calls. **Changed**: the everyday default moves from gemini-flash
+  direct to DeepSeek via OpenRouter — one prepaid pool, visible in `ko billing`.
+- LLM calls report their cost on stderr — `[model · 66→21 tok · $0.0012]` —
+  using OpenRouter's *actual* billed cost when available (genai-prices estimate,
+  marked `~`, otherwise). Covers `ko llm`, agents, `ko brief`, `yt -s`; `sessions
+  summarize` totals its spend.
+- `ko refs` — manage the `~/code/refs` reference-repo folder: bare = parallel
+  `--ff-only` pull of all clones (only moved HEADs printed), `setup` bootstraps
+  the baked list + `~/.config/ko/refs.txt` on any machine, `add <url>` clones and
+  remembers, `list` = TSV. `ko doctor` shows the folder's disk footprint and
+  flags clones over 500MB.
+- Opt-in LLM telemetry: `[telemetry] enabled = true` + `POSTHOG_API_KEY` sends
+  provider-agnostic traces (model/tokens/cost) to PostHog via pydantic-ai OTel
+  instrumentation; off by default, metadata-only unless `include_content = true`.
+- `[llm] model` and `[agents] model` in config.toml — the default model now has
+  a config home (env var still wins); `ko doctor` gains an effective-settings
+  footer (each value's source: env/config/default, plus the three dirs) and
+  live-checks the OpenRouter key via `/credits` — proves it works and shows
+  credits left, without spending tokens.
+
+### Fixed
+
+- A malformed `config.toml` now warns loudly (every command + doctor) instead of
+  silently ignoring all keys and settings; README documents the three dirs.
+
+## [2026.7.9] - 2026-07-09
+
+First tagged release — everything since the repo started (2026-04). `ko` is an
+opinionated personal CLI of thin SDK wrappers, built to be driven by humans and
+AI agents over bash: `--help` is the contract, stdout is data, exit codes mean
+things, `--json` where structure matters.
+
+### Added
+
+- **Read anything**: `ko fetch <url>` — articles, PDFs, arxiv, YouTube, DOIs →
+  markdown, with Wayback/archive.today fallbacks (bare `ko <url>` routes here);
+  `ko doc <file>` — PDF/Office/image → text, local; `ko yt <url>` — transcripts,
+  `-s` summarizes.
+- **Papers**: `ko papers` — cross-publisher search + citation graph via OpenAlex
+  (`search|get|cites|refs|similar`, no key); `ko arxiv` — relevance-ranked search
+  + paper → markdown via arxiv2md; `ko hf` — HF Daily Papers.
+- **News & social**: `ko hn` — `top` (incl. `--now` live front page), `search`,
+  `item` comment trees; `ko x` — search (incl. full-archive), lists by
+  name/id/URL, user timelines; `ko tt` — TickTick lists, read-only.
+- **Google** (one OAuth token, multi-account): `ko gsheets` read & write with a
+  clobber guard; `ko gdocs` Markdown ↔ Docs + comments; `ko cal` agenda/find/add;
+  `ko gmail` read-only search/view/thread.
+- **AI**: `ko llm` — one-shot, stdin-aware, `-m` any model whose provider key is
+  set; `ko agent research|tv` — pydantic-ai agents with saved, resumable
+  sessions; `ko agent sessions` — listing with `--tag`/`--search` filters over an
+  LLM-built SQLite index (`sessions summarize`); `ko brief` — morning brief;
+  `ko prompt` — kickoff briefs; `ko mcp` — inspect/call/overview/auth-info for
+  MCP servers.
+- **Publish**: `ko publish` — folder → Cloudflare static site (scaffolds,
+  optional PIN gate, custom domains).
+- **Meta**: `ko doctor` setup status; `ko billing` OpenRouter credits; `ko logs`
+  local command log (no args/secrets); `ko tv` ratings + where to stream.
+- **Agent contract** (AGENTS.md): stdout=data / stderr=notes, exit 0/1/2,
+  `--json` errors as `{error, code}`, bare-arg shortcuts.
+
+[Unreleased]: https://github.com/khalido/kotools/compare/v2026.7.14...HEAD
+[2026.7.14]: https://github.com/khalido/kotools/compare/v2026.7.9...v2026.7.14
+[2026.7.9]: https://github.com/khalido/kotools/releases/tag/v2026.7.9
+

kotools-2026.7.14/CLAUDE.md

@@ -0,0 +1,108 @@
+# ko
+
+Ko's personal opinionated CLI. Thin wrappers around SDKs I use often, built so both I and AI agents (Claude Code via bash) can lean on it.
+
+## Subcommands
+- `ko exa search|get` — Exa semantic web search + URL → markdown
+- `ko arxiv search|fetch` — arxiv search (**relevance-ranked**; `--recent` = newest-first for browsing a `cat:cs.LG`) + paper-to-markdown. For cross-publisher relevance, prefer `ko papers search`.
+- `ko gsheets info|tabs|get|find | set|put|header|add-tab|new|clear|auth` — read **& write** Google Sheets via OAuth. Accepts a URL or bare ID. Reads use the readonly scope; writes need read+write (`ko gsheets auth` grants it by default). Shape-aware writers (`set`/`put`, library `write_values`/`write_ranges`/`write_df`) refuse to clobber non-empty cells unless `--overwrite` — the error lists them. `write_df` duck-types polars **or** pandas (neither is a dep). Generic port from the NibbleEdge `ne` tool; no project-specific shortcuts.
+- `ko doc <file>` — PDF/Office/image → plain text via liteparse (local, no models). Bare shortcut: `ko <file>` routes here when the arg is an existing file.
+- `ko fetch <url>` — URL → markdown, deterministic routing: arxiv→arxiv2md, YouTube→transcript (via `ko yt`; fails loud when captions are off — no silent fall-through to the useless description page), PDF→~/Downloads+liteparse (`--no-save`), doi.org→OA copy, else trafilatura, dead/empty→Wayback (`--archive` forces). Bare shortcut: `ko <url>`.
+- `ko yt <url|id>` — YouTube → transcript via **youtube-transcript-api** (free, keyless, no video download; prefers manual subs over auto-captions). Plain text with paragraph breaks (>2s gaps); `--json` = `[{start, text, duration}]` snippets; `-s/--summarize` = one-shot LLM summary (`-m` overrides model). Accepts watch/youtu.be/shorts/live/embed URLs or a bare 11-char id. Stderr notes the ~token count. Future fallback for caption-less videos: Gemini native video understanding (URL as `file_data`, fractions of a cent) — package-first because it's free.
+- `ko llm "<prompt>"` — one-shot LLM, stdin-aware, never has tools. Default = the **basic tier** (deepseek-v4-flash; `KO_DEFAULT_MODEL` → `[llm] model` → `[llm] basic` → baked; `-m` per-run, tab-completes models whose env key is set). Prints a cost note to stderr after each call. OpenRouter's catalog is fetched from its public `/models` and cached at `~/.cache/ko/openrouter_models.json` (24h TTL) so `-m openrouter:<slug>` completes.
+- `ko models [--refresh]` — list model strings usable with `-m`, one per line. `--refresh` force-refetches the OpenRouter catalog past the 24h cache (for grabbing a just-released model).
+- `ko brief [--raw] [-m model]` — morning brief: deterministic gather (today's calendar + unread gmail + `hn top` + `hf top`, each best-effort — an unconfigured source becomes a note, never a crash) → **one** cheap-model synthesis (scannable markdown: what's on, mail that matters, the 2-3 items worth a click). `--raw` prints the gathered sections and skips the LLM entirely. A scripted pipeline, NOT an agent loop. ⚠️ deliberate: sends calendar/email content to the configured LLM (noted in help).
+- `ko logs [--n] [--errors] [--json]` — recent local command logs. ko writes **one structured JSON event per command** via **loguru** (`~/.local/state/ko/logs/ko.jsonl`, serialized JSONL, weekly rotation: cmd label, duration, exit code, error type — **no args/secrets**, following PostHog's "one wide event per request"). loguru's default stderr sink is removed (ko's own output stays clean), and its sink model makes adding a sync trivial — a PostHog sync is just another `logger.add(sink)`. On by default; `[logs] enabled = false` (or `[logs] level = "debug"`) in config.toml. Best-effort — never breaks a command.
+- `ko billing` — read-only balance + recent usage across paid services. v1: OpenRouter credits ($ remaining of total purchased; today/week/month usage). `--json`. One function per provider in `billing.py` (degrades gracefully like `ko doctor` — a missing key is a note, not a crash). pydantic-ai has no account-billing API, so we hit each provider's REST API directly.
+- `ko hn top|search|item` — Hacker News via Algolia (no auth). `top` = hckrnews-style top 10/20 by points (`--now` = the live front page via Algolia's `front_page` tag — complementary: the tag only marks the *current* page, `top` does past windows); `search` defaults to last 12 months; `item` = story + comment tree as text.
+- `ko hf top|search|info|get` — Hugging Face paper pages (no auth). `top` = Daily Papers in HF's *trending* order (upvote-driven with recency decay — not strictly upvote-sorted); `search` = semantic; `info` = metadata incl. github/models/datasets; `get` = paper as markdown. Same ids as arxiv — composes with `ko arxiv fetch`.
+- `ko papers search|get|cites|refs|similar` — cross-publisher literature scouting via **OpenAlex** (no key; `mailto` polite pool; its `oa_url` IS Unpaywall's data). Takes DOIs, arxiv ids, or OpenAlex W-ids interchangeably. `search` = all publishers (TSV: year, cites, oa_status, title, doi, journal); `get` = full text via the OA copy + fetch pipeline, else a metadata card (authors, journal, S2 tldr, abstract reconstructed from `abstract_inverted_index`) — paywalled ≠ error; `cites`/`refs` = citation graph, most-cited first; `similar` = S2 recommendations (needs free `S2_API_KEY` — keyless S2 429s immediately, so tldr/similar degrade gracefully without it). Plus DOI routing in `ko fetch`: doi.org URLs resolve the OA copy first instead of 403ing on publisher landing pages. Design: `docs/papers-cli-design.md`. Known wart: a few OpenAlex DOI records are bad merges (wrong title, right abstract) — `search` for the title and use the W-id when a record looks off.
+- `ko tv <title>` — movie/TV rating + overview + regional watch providers (`TMDB_READ_ACCESS_TOKEN`, free; AU default, `--country`).
+- `ko x search|list|user|lists` — X via the official XDK (`X_BEARER_TOKEN`). `list <name|id|url>` = recent posts from a list (name = one of mine; a bare list id or `x.com/i/lists/<id>` URL reads **any public list**); bare shortcut `ko x ai` ≡ `ko x list ai`. `user <@handle|url>` = one user's timeline (further back than search's ~7-day window). `lists` = my lists, biggest first, with descriptions + member counts (owned lists; *followed* lists need a higher tier). `search` ≤7 days uses the recent index; `--days >7` switches to **full-archive** (years back); `--list <name|id|url>` scopes a search to a list (`ko x search "claude code" --list ai --days 90`). **Likes/bookmarks/home-timeline aren't available** — they need OAuth user-context, not the app-only bearer (which also can't self-identify, so set your handle via `[x] handle` in config.toml or `KO_X_HANDLE`; default `ko`). **Cost**: X is pay-per-use since 2026-02 (prepaid credits) — ~$0.005/post read (owned reads like `lists` $0.001), 2M reads/mo cap; so `ko x list --n 20` ≈ $0.10, `ko x lists` ≈ a cent. Full-archive/big `--n` cost more — mind them. [pricing](https://docs.x.com/x-api/getting-started/pricing)
+- `ko tt lists|items <list>` — TickTick (read-only) via its hosted MCP (`TICKTICK_API_KEY`, env or config.toml). `lists` = your lists; `items <list>` = open tasks (name substring or id). Bare shortcut `ko tt <list>` ≡ `ko tt items <list>`. Read-only by design — manage tasks in TickTick itself. We're an MCP *client* here (`mcp` SDK over Streamable HTTP).
+- `ko gdocs get|push|replace|append|comments|reply|shade-table` (alias `ko gd`) — Markdown ↔ Google Docs via OAuth (shared token; Docs + Drive `drive.file` scopes). `get` = Doc→Markdown (Drive export, tables included, pipe it); `push <file> [doc]` = Markdown→new Doc, or update an existing one in place (shows a diff + confirm, `--force` skips); `replace`/`append` = in-place edits; `comments`/`reply` = Drive comments; `shade-table` = header/totals cell shading (the bit Markdown can't do). What converts: see `docs/gdocs-markdown.md`.
+- `ko cal [agenda] | day | find | add | cals` — Google Calendar agenda + quick-add (shared token; `calendar.readonly` + `calendar.events`). Bare `ko cal` = next 7 days across calendars (`--days`/`--today`); `find <text>` = events matching a title substring, forward or `--past` ('when was my last X'; borrowed from chota-bot); `add "title" <when>` accepts ISO / today / tomorrow (no date lib). Times in `[cal] timezone` (default Australia/Sydney). Minimal — no edit/delete/RSVP.
+- `ko gmail [recent] | search | from | view | thread` — read Gmail (**read-only**; shared token, `gmail.readonly`). Bare `ko gmail` = recent inbox, one line/msg (id, date, from, subject + snippet; `-n`, `--unread`); `search "<query>"` passes Gmail's own syntax verbatim (`from:`/`newer_than:7d`/`is:unread`); `from <who>` shortcut; `view <id> [--full]` = one message; `thread <id> [--full]` = the whole conversation, oldest first. `--json` everywhere. Concise on purpose (vs the verbose Gmail MCP). No send/labels/drafts.
+- `ko publish new <dir> [--md|--bare|--hono]` / `ko publish preview [dir]` / `ko publish [dir]` / `ko publish list` — publish a folder to Cloudflare. `preview` runs `wrangler dev` locally (real http; `file://` breaks ES modules + fetch). `new` scaffolds: default static (Tailwind+Alpine), `--md` (markdown doc site — generic `?page=` shell, README hub, sidebar TOC), `--bare` (just a CLAUDE.md of hints). Bare `ko publish [dir]` deploys; the folder's own `wrangler.jsonc` makes the name sticky → re-publish overwrites the same URL. Custom domain defaults to `<name>.khalido.dev` (`DEFAULT_DOMAIN`; override via `[publish] domain`). Creds: `KO_CLOUDFLARE_API_TOKEN`/`KO_CLOUDFLARE_ACCOUNT_ID` → config.toml → standard `CLOUDFLARE_*`. **Won't silently take over** an existing Worker name (checks the account; `--force` to override). Needs **wrangler** (pinned repo-local via `npm install`; `KO_WRANGLER` → `node_modules/.bin` → PATH → `npx wrangler@4`). `--hono` adds a Hono worker (API routes, D1/R2, optional `--pin` gate, cached `/api/data`); `ko publish --pin new` rotates the gate PIN. Cloudflare Access private pages + AI on sites deferred — see `docs/publish.md` / `docs/publish-ai.md`.
+- `ko ai "<prompt>"` — **the default agent**: all Layer-1 toolsets (web, papers, HN, movies, read-only ~/code files) + its own memory (`memory/ai/`), medium tier, hard cap 30 model requests/run. One-shot or REPL, `-m`/`-r` as usual. Explicit command on purpose — intelligence is opt-in; a typo'd subcommand must never silently spend tokens. Specialists stay under `ko agent`.
+- `ko agent research|repo|tv|sessions` — pydantic-ai agents. `research` (web+papers+HN, smart tier), `repo` (repo-explorer: "how does repo X do Y" over ~/code read-only — own repos + `refs/`; knows the refs/CLAUDE.md map, injected fresh per run; cites file:line; basic tier — a full exploration costs ~$0.0005), `tv` (movies/TV in AU, cheap model), no prompt = interactive REPL, `-m` overrides model, `-r <id>` resumes a saved session. `sessions` lists saved sessions (title/summary/tags once summarized; `--tag`/`--search` filter); `sessions summarize [--n] [-m]` = cheap-model title + one-line takeaway + tags per session into the SQLite index. **`ko a …` is a hidden shorthand for `ko agent …`** (e.g. `ko a research "…"`). See Agents below.
+- `ko mcp inspect <name|url>` / `ko mcp call <name|url> <tool>` / `ko mcp overview <name|url>` / `ko mcp servers` — investigate/use MCP servers (ko is itself an MCP client). `overview` runs a pydantic-ai agent that connects (via `load_mcp_toolsets` — same mcp.json shape) and summarizes what the server is for + what's useful (reads only, won't call mutating tools; cheap model, `-m` to override) — handy for sizing up your own dev server. Target a **server name** from `~/.config/ko/mcp.json` (the standard `{"mcpServers": {...}}` shape — copy-paste-compatible with Claude Code/opencode; supports both `url`+`headers` and stdio `command`/`args`; secrets use `${ENV_VAR}` placeholders expanded from env/config at load, so the file is safe to check in) **or a raw URL**. `inspect` lists tools + resources + prompts (+ capabilities + the server's instructions); `--tool <name>` dumps that tool's full JSON input schema. `call` invokes a tool (`--arg k=v`). `servers` lists the registry. **`auth-info <url>`** is spec-aware (RFC 9728 + 8414): it walks a remote server's OAuth discovery surface *before* connecting — protected-resource + auth-server metadata, scopes, endpoints, DCR — and flags anything malformed (e.g. HTML where JSON should be); exit 1 on problems. `-H` overrides/adds headers, `--json` everywhere. **On failure** it prints the server's real HTTP status + body, the `WWW-Authenticate` header (the server saying *why*), and — if a Bearer token was passed — the token's decoded `aud/iss/scope/exp` claims (a wrong audience is otherwise invisible). (A 503 "not configured" = server-side, not your client.) A verb takes a name/url = client; a future `ko mcp serve` (no target) = ko as server. Listings to stdout, banner to stderr.
+- `ko refs [pull] | setup | add <url> | list` — manage `~/code/refs` (read-only reference clones for ideas/patterns; `[refs] dir`/`KO_REFS_DIR` overrides). Bare `ko refs` = parallel `git pull --ff-only --prune` (worktree groups serial; prints only repos whose HEAD moved as `repo: old -> new`; confirm on TTY only — pipes/agents never prompt). `setup` = idempotent bootstrap: dir + the folder CLAUDE.md (written only if missing — it accumulates takeaways) + clones the **baked list** (`refs.BAKED_REPOS`, the opinionated default) plus `~/.config/ko/refs.txt` (machine-local, one URL/line, dotfile-synced). `add <url> [--note]` clones + stubs a CLAUDE.md entry + appends to refs.txt so setup restores it anywhere. The dir itself is the truth for pull/list; the lists exist only for bootstrap. Pull logic ported from dotfiles/gitupdateall.py. my opinionated "how I build X" notes, pulled by name to load into an agent ("get `ko prompt sveltekit-app`") or copy-paste (`--copy`). Bare `ko prompt` lists (TSV: name, description); `ko prompt <name>` prints the markdown. Briefs ship in `src/ko/prompts/*.md`; add or override per machine in `~/.config/ko/prompts/*.md` (a user file wins by name). A meta-layer on top of generic llms.txt — my picks + the gotchas, with links out to the canonical docs. The tool is just a thin loader; the content is hand-written.
+
+## Principles
+- **Opinionated, not generic.** Do what I actually do, the way I do it. Defaults match my habits.
+- **Thin wrappers.** Each module (`exa.py`, `arxiv.py`, `gsheets.py`) is a self-contained domain module over one upstream SDK. Keep parameter names matching the SDK so agent-written code composes with upstream docs.
+- **Help lines carry their weight.** Every subcommand and flag has a concise, useful `help=`. Assume humans and agents skim `--help` and nothing else.
+- **Human-readable and pipeable by default.** Plain text / TSV. Add `--json` when structure matters. Errors to stderr. `typer.Exit(0)` for "no results" (not an error).
+- **One file per domain.** `cli.py` registers subapps; each module owns its logic. No shared `utils/` until three things need the same thing.
+- **`--help` is the contract.** Machine-friendly output is a design constraint, not an afterthought.
+
+## Stack
+- Python 3.14, `uv`, `ruff`, `pytest`
+- `typer` for the CLI
+- Module-local `@dataclass` for structured returns
+- `@lru_cache` for credential/service singletons
+
+## Releasing
+`/release` (repo skill, `.claude/skills/release/`) — CalVer (`YYYY.M.D`, PEP 440), rolls commits
+into `CHANGELOG.md` (canonical) + a GitHub release (derived), fact-checked by a fresh subagent,
+human approval before tag/publish. WORKLOG = dev journal; CHANGELOG = release-grained facts.
+
+## Adding a subcommand
+1. `src/ko/<thing>.py` — thin SDK wrapper + dataclass result types
+2. `src/ko/cli.py` — Typer subapp + commands, every flag with `help=`
+3. `tests/test_<thing>.py` — smoke test; skip live calls if env var / token missing
+4. Deps in `pyproject.toml` under `[project].dependencies`
+
+## Agents (`src/ko/agents/`)
+- **Tools declared once** as `FunctionToolset`s in `_toolsets.py` (`web` = exa+fetch, `papers` = arxiv+hf+OpenAlex citation-graph [search/get/cites/refs], `news` = hn [top/front_page/search/discussion — search passes through since_months/by_date/min_comments], `tmdb`, `files` = read-only list_dir/read_file/grep/glob over `~/code` for the repo agent (names/shapes match Claude Code's Read/Grep/Glob priors; listings show ≈token sizes so the model judges read-whole vs grep-first) — `agents/_files.py`: containment resolves symlinks BEFORE `is_relative_to` (thinker's `resolve_in` + harness's TOCTTOU ordering), read-only by construction (no write tools exist), capped output, ripgrep search, ModelRetry-with-listing on misses); agents compose the subset they need. **Docstring-as-contract**: a tool docstring must state any silent filtering default of the underlying function (date windows, result caps, sort orders, index coverage) — the agent-side twin of "--help is the contract". The research agent's instructions encode a snowball workflow (seed → cites/refs → saturation); see `ko prompt research-papers` for the human-facing version. A toolset is stateless and shared by reference; each carries its own `instructions=`.
+- **One file per agent** (`ai.py` = the default, `research.py`, `repo.py`, `tv.py`): ~15-50 lines — `Agent(model, instructions=..., toolsets=[...])`. Per-agent default model (research = capable, tv = cheap); `KO_AGENT_MODEL` → `[agents] model` in config.toml overrides the default at import, `-m` overrides per run (passed to the run call, not baked into the Agent).
+- **`_shared.py`** — generic run/stream/repl over any Agent; streams pretty markdown to a TTY, plain text when piped.
+- **Resilient tools** — each wraps its call in `_try`, returning an error note instead of raising, so one flaky source (rate limit, timeout) doesn't abort a run; the model routes to another source.
+- **Sessions** (`src/ko/sessions.py`) — every turn dumps `all_messages()` (full tool trace) to `~/.local/state/ko/sessions/<id>.json` via pydantic-ai's `ModelMessagesTypeAdapter`. Resume with `-r <id>`; list with `ko agent sessions`. Flat files in state (not config — generated, not hand-edited).
+- **Memory** (`agents/_memory.py`, per docs/memory.md "v1 first") — research + repo agents keep a writable workspace at `~/.local/state/ko/memory/<agent>/`: `memory.md` is the anchor (head-injected every run, pin marker `<!-- ko:pin-end -->` protects foundations from truncation, newest tail survives), plus any other .md notes the agent creates (listed in the injection, read on demand). Tools: `read_note`/`append_memory`/`edit_note` (uniqueness-guarded)/`write_note` (anchor protected from overwrite) — scoped to the agent's own dir, .md-only; the code `files` toolset stays read-only. Shared `~/.config/ko/memory.md` (dotfile-synced user context) injects into all memory agents; hand-edited in v1. Curation rules live in the toolset instructions (thinker's discipline).
+- **Session index** (`~/.local/state/ko/ko.db`, SQLite) — `ko agent sessions summarize` writes one row per session (title ≤8 words, one-sentence takeaway, 3-5 tags) via a cheap-model structured-output call; `sessions --tag/--search` filter on it. The JSON files stay the source of truth — the DB is a rebuildable projection ("the log is the agent"). The model sees a **deterministic digest**, not the raw log: Q&A verbatim, each tool call compressed to one `[tool: name(args…) → return…]` line; trimming drops tool lines first, middle turns next, never the first prompt or final answer.
+- **No YAML agent-spec**: pydantic-ai specs can't reference custom toolsets, so code factories win. Capabilities (lifecycle hooks) are deferred until we need request/tool interception.
+
+## Adding an agent
+1. Add/extend a `FunctionToolset` in `agents/_toolsets.py` if new tools are needed.
+2. `agents/<name>.py` — `Agent(model, instructions=..., toolsets=[...])` + thin `run`/`repl` binding `_shared`.
+3. Export in `agents/__init__.py`; add a `@agent_app.command` in `cli.py`.
+4. **Close the loop — the toolset eval method** (house method for any toolset work, proven
+   2026-07-10 on the files tools): after building or changing tools, (a) run the agent on 1-2
+   real questions (cheap — cents), (b) have a subagent READ THE SESSION TRACES
+   (`~/.local/state/ko/sessions/` keeps full tool calls/returns): flailing? repeated calls?
+   params the model *invented*? (that's a tool gap — the files `grep` gained `limit` exactly
+   this way), (c) a separate fresh-context adversarial review of the tool code. Each pass
+   catches what the others can't: traces find ergonomics, review finds crashes, and
+   prior-alignment (do names/shapes match what models are trained on — Claude Code's
+   Read/Grep/Glob, rg semantics?) finds friction. The tools' heaviest user is the model —
+   let it review them.
+
+## Google auth (gsheets)
+- OAuth user flow (desktop app). First command triggers browser consent, token cached at `~/.local/state/ko/google_token.json`.
+- `ko gsheets auth` (or `gdocs`/`cal`/`gmail auth` — same flow) grants **one read+write token per account** covering Sheets + Docs + Calendar (write) and Gmail (**read-only**). **No Drive scope** — can't browse Drive, only docs/sheets by ID. Reads use the narrower readonly scopes; the one token serves all. `--readonly` narrows; upgrade an existing token with `auth --logout` then `auth`. Adding another API later = one re-consent.
+- Personal Gmail: an **External** app left in *Testing* expires the refresh token after **7 days** — set the consent screen to **Internal** (Workspace) or **Publish** the app. The token file is portable across machines (copy it, or share via `KO_STATE_DIR`).
+- **Multi-account.** Active account = `-a/--account` (any `ko gsheets` cmd, sets `KO_GOOGLE_ACCOUNT`) → `[google] account` → `"default"`. Per-account token `google_token_<account>.json` (legacy `google_token.json` for `default`); per-account client `google_client_<account>.json` falls back to the shared `google_client.json` (only needed when one OAuth client can't authorize an account, e.g. a Workspace-Internal consent screen + a personal Gmail). `ko gsheets accounts` lists authed accounts. All paths via `google_auth.token_file()/client_file()/active_account()`.
+- **Scope is per-API, not per-folder.** OAuth grants the tool *your* access within the scope (now Sheets only). It can't be limited to specific folders/files; true per-file scoping needs a service account (share files with its email) — a separate model not used here.
+- Prereq: OAuth 2.0 Desktop client JSON at `~/.config/ko/google_client.json` (or set `KO_GOOGLE_CLIENT_FILE`). Full setup in README.
+
+## Directories (`src/ko/dirs.py`)
+- `~/.config/ko/` — user-editable config, dotfile-sync-safe (`KO_CONFIG_DIR`). `config.toml` `[keys]` table supplies API keys/tokens as a fallback to env vars (env always wins); loaded into the environment at startup so SDKs that read `os.environ` directly pick them up. Non-secret settings follow the same chain via `config.setting()` (env → config.toml → baked default): `[llm] model`, `[agents] model`/`tv_tastes`, `[x] handle`, `[google] account`, `[cal] timezone`, `[publish] domain`, `[logs]`, `[telemetry]`. **Model tiers** (`llm.model_for(tier)`): `[llm] basic/medium/smart/ultra`, override per-tier in config.toml. Money-first picks (prices $/M in+out, 2026-07-10): basic = deepseek-v4-pro $0.43/$0.87 (llm default, tv, summarize, brief; deepseek-v4-flash is 5× cheaper if basic feels rich), medium = glm-5.2 $0.53/$1.67, smart = `~x-ai/grok-latest` $2/$6 (alias, tracks newest grok; research agent), ultra = gpt-5.6-sol $5/$30 (reserved — nothing uses it by default). Cheap Chinese models carry basic/medium; ultra is the only expensive one. All via OpenRouter, so `ko billing`/doctor's credits line watches the whole spend. **Per-run cost notes**: every LLM call prints `[model · in→out tok · $x]` to stderr (`llm.run_cost`/`last_cost`) — OpenRouter's actual billed cost from `provider_details['cost']` when present, else a genai-prices estimate marked `~` (which misses brand-new models → tokens only). `ko doctor` shows each key's source (env / config / missing), an effective-settings + tiers + dirs footer, warns on malformed config.toml, and live-checks the OpenRouter key via `/credits` (proves the key + shows $ left, no tokens spent).
+- **`[telemetry]` — opt-in LLM telemetry** (`telemetry.py`, off by default): `enabled = true` + `POSTHOG_API_KEY` instruments all pydantic-ai agents (`Agent.instrument_all` → OTel `gen_ai.*` spans → PostHog's AI OTLP endpoint) — provider-agnostic, covers llm/agent/brief/summarize. `include_content = false` by default (metadata only — brief pipes gmail/cal through the LLM; don't mirror that to PostHog). Disabled path is one dict lookup, no OTel imports. GH issue #2.
+- `~/.local/state/ko/` — tokens + id caches, never synced (`KO_STATE_DIR`); old config-dir state files auto-migrate
+- `~/.cache/ko/` — disposable (`KO_CACHE_DIR`)
+- `ko doctor` shows per-tool setup status (keys set, binaries found, auth done)
+
+## Agent notes
+- **`AGENTS.md`** at the repo root is the agent-facing contract (output/exit-code/`--json` conventions, bare-arg shortcuts). Keep it in sync with the bullets below.
+- `ko --help` / `ko <cmd> --help` are the contract
+- `ko gsheets get ... ` defaults to TSV — `cut -f N` safe
+- `--json` on most read commands → a JSON array of objects (a few return a single object; `gsheets get` is a 2D array). Under `--json`: errors are `{error, code}` on stderr, empty results are `[]` on stdout. stdout = data, stderr = notes/errors.
+- Exit codes: `0` success (incl. empty results), `1` runtime error, `2` usage error
+- No interactive prompts inside commands (except the one-time OAuth browser popup)
+
+## Not yet
+- **MCP server (`ko mcp`)** — `mcp_server.py` is scaffolded but not wired; expose the same module functions over MCP.
+- **tmdb lists by id** in config; `ko ai` skills/capabilities + mcp.json toolsets (registry ready, wire when needed). Full candidate list + priorities in `docs/ideas.md`.
+- **Gemini video fallback for `ko yt`** — caption-less videos via native video understanding (URL as `file_data`); build when the no-captions case actually bites.