@headband/cli 0.1.0 → 0.3.0

package/README.md

@@ -1,33 +1,421 @@
 # @headband/cli
 
-Interactive CLI for setting up [Headband](https://headband.dev) search in any project.
-
-## Usage
+Command-line tools for [Headband](https://headband.dev): project setup, login,
+API keys, indexes, documents, search, tasks, dedicated engines, schema
+migrations and Sync. Every command works for a person at a terminal and for a
+script or coding agent with no terminal at all.
 
 ```bash
+npx @headband/cli --help
 npx @headband/cli init
+npx @headband/cli login
+```
+
+## Contents
+
+- [Global options](#global-options)
+- [Authentication](#authentication)
+- [Scripts and agents](#scripts-and-agents)
+- [Exit codes](#exit-codes)
+- [Command reference](#command-reference)
+- [What `init` creates](#what-init-creates)
+
+## Global options
+
+```text
+--token <key>      API key for this call. Wins over HEADBAND_API_KEY and stored logins.
+--profile <name>   Stored login to use (else HEADBAND_PROFILE, else the current profile)
+--url <url>        Headband URL (else HEADBAND_URL, else the profile's, else https://headband.dev)
+--json             Print one JSON document on stdout; errors print one JSON object on stderr
+--yes, -y          Answer yes to confirmations (required when stdin is not a terminal)
+--help, -h         Show help for the CLI or a command
+```
+
+## Authentication
+
+### Interactive login
+
+```bash
+headband login [--url <url>] [--profile <name>] [--json]
+headband logout [--profile <name>] [--yes] [--no-revoke] [--json]
+headband whoami [--json]
+headband profiles list [--json]
+headband profiles use <name>
+```
+
+`login` runs Headband's device flow: it prints a code and a link, opens your
+browser, and waits while you approve it in the dashboard. It saves the key it
+receives as a named profile and makes that profile current. The name defaults
+to `default` for headband.dev and to the host name for any other URL, such as
+`localhost-3000`. The full key is never printed; output shows only its prefix.
+
+Log in to several Headband URLs or organizations and switch between them:
+
+```bash
+headband login --profile prod
+headband login --url http://localhost:3000 --profile local
+headband profiles list
+headband profiles use prod
+headband indexes list --profile local   # one command, without switching
+```
+
+`logout` revokes the profile's key on the server, then removes it from this
+machine. It asks first at a terminal. `--yes` revokes without asking.
+`--no-revoke` removes the local copy and leaves the key working. If the key is
+already gone on the server, logout still succeeds; any other failure keeps the
+local login so you can retry.
+
+Credentials live in `~/.config/headband/credentials.json`, or
+`$XDG_CONFIG_HOME/headband/credentials.json`, with file mode `0600`. A file
+written by CLI 0.2.0 or earlier (profiles keyed by URL) still works: its
+profiles get names on first read, and the file is saved in the new format the
+next time the CLI writes it.
+
+### Tokens and environment variables
+
+A script, CI job or agent does not need to log in. Give it a key:
+
+| Variable           | Flag        | Meaning                                                |
+| ------------------ | ----------- | ------------------------------------------------------ |
+| `HEADBAND_API_KEY` | `--token`   | The API key to send. Used instead of any stored login. |
+| `HEADBAND_URL`     | `--url`     | The Headband URL.                                      |
+| `HEADBAND_PROFILE` | `--profile` | Which stored login to use.                             |
+
+Precedence, highest first:
+
+- **Key:** `--token`, then `HEADBAND_API_KEY`, then the stored profile.
+- **URL:** `--url`, then `HEADBAND_URL`, then the profile's URL, then
+  `https://headband.dev`. With a token, the profile's URL counts only when you
+  name the profile with `--profile`.
+- **Profile:** `--profile`, then `HEADBAND_PROFILE`, then the current profile,
+  then the only stored profile when there is one.
+
+A stored key is only ever sent to the URL it was issued for. `--url` with no
+`--profile` picks the stored profile for that URL; `--url` with a `--profile`
+for a different URL is a usage error. A `--profile` name that does not exist is
+an error, never a silent fallback.
+
+```bash
+export HEADBAND_API_KEY=hb_adm_...
+export HEADBAND_URL=https://headband.dev
+headband indexes list --json
+```
+
+Which key to use: an admin key from `headband keys create <name> --type admin`
+covers indexes, documents, search, engines and Sync. Managing keys, projects
+and schema state needs a key minted by `headband login`, because only those
+carry `keys.manage`, `projects.manage` and `schema.manage`.
+
+## Scripts and agents
+
+The CLI never waits for input it cannot get.
+
+- **No hanging prompts.** A command that would ask for confirmation fails at
+  once with exit code `2` and code `confirmation_required` when stdin is not a
+  terminal. Pass `--yes` to confirm. `headband init` does the same without
+  `--yes`.
+- **One JSON document.** With `--json`, stdout carries exactly one JSON
+  document and progress lines go to stderr, so `headband ... --json | jq` is
+  safe. `docs export` is the exception by design: its output is the documents.
+- **JSON errors.** With `--json`, a failure prints one line to stderr:
+
+  ```json
+  { "error": "Index not found", "code": "index_not_found", "status": 404, "exitCode": 5 }
+  ```
+
+  `code` is the API's error code when there is one, else a CLI code such as
+  `usage_error`, `not_authenticated`, `confirmation_required` or
+  `network_error`. `status` is the HTTP status, or `null` when no request was
+  made.
+
+- **Branch on exit codes**, not on message text. See the table below.
+
+## Exit codes
+
+| Code  | Meaning          | Typical cause                                                                                                   |
+| ----- | ---------------- | --------------------------------------------------------------------------------------------------------------- |
+| `0`   | Success          |                                                                                                                 |
+| `1`   | Error            | Anything not listed below; also `schema diff` when changes are pending                                          |
+| `2`   | Usage            | Unknown command or option, missing or malformed value, HTTP 400/413/422, confirmation needed without a terminal |
+| `3`   | Authentication   | No key found, a `--profile` that does not exist, HTTP 401                                                       |
+| `4`   | Permission       | HTTP 403, or 402 (the plan does not allow it)                                                                   |
+| `5`   | Not found        | HTTP 404 or 410, `profiles use` with an unknown name                                                            |
+| `6`   | Conflict         | HTTP 409, such as deleting a project that still has indexes                                                     |
+| `7`   | Rate limited     | HTTP 429                                                                                                        |
+| `8`   | Server or engine | HTTP 5xx, or the URL could not be reached                                                                       |
+| `130` | Interrupted      | Ctrl+C                                                                                                          |
+
+## Command reference
+
+### Setup
+
+```bash
+headband init [--yes]
+```
+
+Asks for your Headband URL, a browser-safe search key (`hb_src_...` only) and
+a default index, installs `@headband/react`, and writes `headband.config.ts`
+plus an example `<HeadbandSearch />` component. `--yes` skips the questions and
+uses defaults. Running `headband` with no command runs `init`.
+
+### Projects
+
+```bash
+headband projects list [--json]
+headband projects create <name> --engine <meilisearch|typesense|elasticsearch|visor> [--json]
+headband projects delete <id> [--force] [--yes] [--json]
+```
+
+`projects delete` refuses a project that still has indexes (exit `6`, code
+`project_has_indexes`). `--force` deletes the project's indexes on its shared
+engine along with it. Even with `--force`, a project with an index on a
+dedicated engine, or an index being moved to one, is refused: delete or move
+those indexes first. Deleting a project also deletes the API keys scoped to it.
+
+### Keys
+
+```bash
+headband keys list [--json]
+headband keys create <name> --type <admin|search> [--project <id>]
+                     [--permissions <a,b,...>] [--expires <30d|ISO date>] [--json]
+headband keys revoke <id> [--yes] [--json]
 ```
 
-This will:
+`keys create` is the only command that prints a raw API key. Store it when it is
+shown: Headband returns raw keys only once.
+
+`--permissions` limits the new key to the permissions you list. You can grant
+only permissions your own key holds, and only ones that fit the key type:
+
+| Type     | Grantable permissions                                                                                                                                                                            |
+| -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
+| `search` | `search`, `analytics.write`                                                                                                                                                                      |
+| `admin`  | `search`, `documents.read`, `documents.add`, `documents.delete`, `indexes.create`, `indexes.delete`, `settings.update`, `analytics.write`, `connectors.write`, `instances.manage`, `sync.manage` |
 
-1. Ask for your Headband URL (default: `https://headband.dev`)
-2. Ask for your search API key (`hb_src_...` or `hb_adm_...`)
-3. Ask for your default search index (e.g. `products`, `movies`)
-4. Detect your package manager (npm, pnpm, yarn, or bun)
-5. Install `@headband/react`
-6. Create `headband.config.ts` with your settings
-7. Create an example `<HeadbandSearch />` component
+Anything else is refused with exit `4` and the names it refused. Without
+`--permissions`, the key gets the type's defaults, cut down to what your key
+holds. `keys.manage`, `projects.manage` and `schema.manage` are never
+grantable here; only `headband login` mints them.
 
-## Options
+`--expires` sets when the key stops working: a duration from now (`12h`,
+`30d`, `2w`, `1y`) or an ISO date (`2027-01-31`). It must be in the future and
+at most one year away. Without it, the key does not expire.
 
 ```bash
-npx @headband/cli init --yes   # Skip prompts, use defaults
-npx @headband/cli --help       # Show help
+headband keys create ci-reader --type admin --permissions search,documents.read --expires 30d --json
 ```
 
-## What gets created
+### Indexes
+
+```bash
+headband indexes list [--json]
+headband indexes create <uid> [--primary-key <pk>] [--engine <engine>] [--instance <id>] [--json]
+headband indexes delete <uid> [--yes] [--json]
+headband indexes settings get <uid>
+headband indexes settings set <uid> --file <settings.json>
+headband indexes stats <uid>
+```
 
-**`headband.config.ts`** -- Search configuration that reads from `NEXT_PUBLIC_HEADBAND_KEY` env var with a fallback:
+`settings get`, `settings set` and `stats` always print JSON.
+
+`--instance <id>` creates the index on one of your dedicated engines (ids from
+`headband instances list`). The index takes that engine's type, so `--engine` is
+optional; if you pass it, it must match.
+
+On the engine, the index has a longer name (`engineIndexUid`, such as
+`hbo_<org>_products`). `indexes create` and `indexes list` show it. Use that name,
+not the uid, with the engine's own client on the instance endpoint.
+
+### Documents
+
+```bash
+headband docs push <file.json|file.jsonl> --index <uid> [--json]
+headband docs export --index <uid> [--out <file>] [--format <jsonl|json>] [--batch <n>] [--json]
+headband docs delete <docId> --index <uid> [--json]
+```
+
+`docs push` accepts a JSON array or a JSONL file and sends documents to
+`/v1/bulk` in batches of 1000.
+
+`docs export` pages through `GET /v1/documents` and writes every document in
+the index, one page in memory at a time:
+
+```bash
+headband docs export --index products > products.jsonl
+headband docs export --index products --out products.json
+headband docs export --index products | head -5
+```
+
+- Without `--out`, documents stream to stdout and nothing else is printed
+  there. Progress for `--out` goes to stderr.
+- `--format` is `jsonl` (one document per line, the default) or `json` (one
+  array). With `--out`, a `.json` file name picks `json`.
+- `--out` writes to `<file>.partial` and renames it when the export finishes,
+  so a failed export never leaves a truncated file under the real name.
+  `--json` with `--out` prints a summary: `{index, out, format, exported}`.
+- `--batch` is the page size. The default is 1000, or 250 for an index on
+  Typesense, which refuses larger pages.
+- Elasticsearch serves at most the first 10,000 documents by paging, so a
+  larger Elasticsearch index fails partway with exit `8`.
+- Export needs an admin key with `documents.read`.
+
+### Search
+
+```bash
+headband search <index> <query> [--limit N] [--filter <expr>] [--json]
+```
+
+Plain output shows a compact table. `--json` prints the raw API response.
+
+### Tasks
+
+```bash
+headband tasks list [--json]
+headband tasks get <id>
+```
+
+`tasks list` prints a table, or the raw response with `--json`. `tasks get`
+prints JSON.
+
+### Engines
+
+Dedicated engines are private Meilisearch, Elasticsearch or Typesense servers for
+your organization, reachable with each engine's own client. They are rolling out
+gradually. Until your organization has access, these commands say so.
+
+```bash
+headband instances list [--json]
+headband instances create <meilisearch|elasticsearch|typesense|visor> [--name <name>]
+                          [--lane <serverless|always_on>] [--memory <512m|1g|...>]
+                          [--region <region>] [--idle-timeout <seconds>] [--json]
+headband instances show <id> [--json]
+headband instances update <id> [--name <name>] [--lane <serverless|always_on>]
+                          [--memory <512m|1g|...>] [--idle-timeout <seconds>] [--json]
+headband instances wake <id> [--json]
+headband instances park <id> [--json]
+headband instances wakes <id> [--limit <n>] [--json]
+headband instances delete <id> [--yes] [--json]
+headband instances connect <id> [--client <node|curl>] [--json]
+```
+
+`update` changes only the settings you pass; `--size` is an alias for
+`--memory`. The engine and region cannot change. A resize applies in the
+background: `instances show` reports the state as "applying changes" until it
+is done. Always-on engines need a paid plan (exit `4` otherwise).
+
+`wakes` lists recent wake-ups, newest first: when each was requested, how it
+woke, the outcome and how long it took to be ready. `--limit` takes up to 200;
+the default is 50.
+
+`connect` prints the endpoint and a snippet for the engine's own client.
+Authenticate with an organization-wide key. The snippets create an index and add
+a document, so run them with an admin key (`headband keys create <name> --type
+admin`); a search key can only search. Project keys do not work on engine
+endpoints. Serverless engines are available for Meilisearch, and for Typesense
+in early access; Elasticsearch runs always on. If a command says the key cannot
+manage engines, run `headband login` again.
+
+### Schema migrations
+
+For projects that define their indexes in `headband.config.ts` with
+`@headband/sync`. These commands read that file from the current directory.
+
+```bash
+headband schema diff [--allow-destructive] [--json]
+headband schema generate [--slug <slug>] [--dir <dir>] [--allow-destructive] [--json]
+headband schema apply [--allow-destructive] [--dir <dir>] [--yes] [--json]
+headband schema status [--dir <dir>] [--json]
+```
+
+- `diff` compares the config with the schema state on the server. Like
+  `git diff --exit-code`, it exits `1` when changes are pending and `0` when
+  in sync. `--json` prints `{inSync, ops}`.
+- `generate` writes a migration file for the pending changes into `--dir`
+  (default `headband/migrations`). `--json` prints `{path}`, with `null` when
+  there was nothing to generate.
+- `apply` runs the migration files not yet applied, in order, backfilling
+  documents through your config's `adapter` when a change needs a reindex.
+  Removing an index needs `--allow-destructive`. `--json` prints
+  `{appliedFiles, appliedOps}`.
+- `status` shows each index's applied version and any drift. `--json` prints
+  `{indexes, localApplied, localHead}`.
+
+Schema commands need a key with `schema.manage`, which only `headband login`
+mints.
+
+### Sync
+
+Sync keeps search indexes in step with a Postgres database. There are two ways
+to run it.
+
+**Managed Sync** runs on Headband. You register a source database and define
+synced indexes; Headband reads the database's changes and keeps the indexes
+current.
+
+```bash
+headband sync source add <name> [--db-url <url>] [--tls-mode verify-full|require] [--ca-file <pem>] [--json]
+headband sync source add <name> --kind batadb --batadb-project <id> --batadb-branch <id> --batadb-compute <id> --accept-broad-role
+headband sync source list [--json]
+headband sync source show <id> [--json]
+headband sync source test <id> [--json]
+headband sync source setup-sql <id> [--provider generic|rds|batadb] [--json]
+headband sync source setup <id> [--owner-url <url>] [--provider generic|rds|batadb] [--yes] [--json]
+headband sync source set-ca <id> (--ca-file <pem> | --none) [--tls-mode verify-full|require] [--json]
+headband sync source remove <id> [--yes] [--json]
+
+headband sync index create -f <file.sync.json> [--json]
+headband sync index list [--source <id>] [--instance <id>] [--json]
+headband sync index show <id> [--version <n>] [--json]
+headband sync index update <id> -f <file.sync.json> [--rebuild] [--json]
+headband sync index validate -f <file.sync.json> [--index <id>] [--json]
+headband sync index preview <id> [--ids <id,id,...>] [-f <file.sync.json>] [--json]
+headband sync index rebuild|pause|resume|check <id> [--yes] [--json]
+headband sync index remove <id> [--drop-index] [--yes] [--json]
+
+headband sync status [--json]
+```
+
+Keep connection strings out of shell history: the source URL can come from
+`HEADBAND_SYNC_DB_URL`, the owner URL from `HEADBAND_SYNC_OWNER_URL`, and a
+BataDB API key from `HEADBAND_BATADB_API_KEY` only. `sync source setup` runs
+Headband's setup SQL on your machine as the tables' owner, in one transaction;
+the owner URL never leaves your machine. Run `headband sync source --help` and
+`headband sync index --help` for the details.
+
+**Self-hosted Sync** runs a worker in your own infrastructure. Your app writes
+changes to an outbox table, and the worker pushes them to Headband.
+
+```bash
+headband sync init [--table <name>] [--write] [--json]
+headband sync worker [--loop] [--budget-ms <ms>] [--batch-size <n>] [--reconcile-every <n>] [--json]
+headband sync worker --scaffold docker [--force] [--json]
+headband sync reconcile [--fix] [--json]
+headband sync retry-dead-letters [--json]
+```
+
+- `init` prints the outbox table's DDL; `--write` also saves it to
+  `headband/outbox.sql`. `--json` prints `{ddl, path}`.
+- `worker` drains the outbox. Without `--budget-ms` it loops until stopped
+  (Ctrl+C or SIGTERM finish the current batch first). With `--budget-ms` it
+  stops after that long; `--json` then prints the run report. In a loop,
+  `--json` sends progress to stderr and prints the report when the worker
+  stops.
+- `worker --scaffold docker` writes `Dockerfile.headband-sync` and prints a
+  compose service for it.
+- `reconcile` compares each index's document count with its source table and
+  reports drift; `--fix` resends a drifted index in full. `--json` prints
+  `{indexes}`.
+- `retry-dead-letters` requeues rows that failed too many times. `--json`
+  prints `{requeued}`.
+
+The worker and `retry-dead-letters` read the outbox database URL from
+`DATABASE_URL`, falling back to `HEADBAND_DATABASE_URL`. The worker and
+`reconcile` need `headband.config.ts` to export an `adapter`.
+
+## What `init` creates
+
+**`headband.config.ts`**: search configuration that reads from the
+`NEXT_PUBLIC_HEADBAND_KEY` env var with a fallback:
 
 ```typescript
 export const headbandConfig = {
@@ -37,18 +425,18 @@ export const headbandConfig = {
 };
 ```
 
-**`components/headband-search.tsx`** (or `src/components/` if your project uses a `src/` directory) -- A ready-to-use search component:
+Do not paste an `hb_adm_` admin key into this flow. The generated config is used
+by a browser component, so the CLI only accepts scoped `hb_src_` search keys.
+Keep admin keys in server-only environments for indexing, settings, and sync
+jobs.
+
+**`components/headband-search.tsx`** (or `src/components/` if your project uses
+a `src/` directory): a ready-to-use search component:
 
 ```tsx
 "use client";
 
-import {
-  headband,
-  HeadbandProvider,
-  SearchBox,
-  Hits,
-  Stats,
-} from "@headband/react";
+import { headband, HeadbandProvider, SearchBox, Hits, Stats } from "@headband/react";
 import { headbandConfig } from "../headband.config";
 
 const client = headband(headbandConfig.url, headbandConfig.searchKey);