@metabase/rde 0.1.0

package/README.md

@@ -0,0 +1,282 @@
+# @metabase/rde
+
+`rde` prepares a Mac or Linux machine for data-engineering work with Metabase and coding agents. One command, `rde init`, walks through everything a working setup needs: the `mb` CLI, the `rde` skill for your coding agents, a running Metabase Enterprise instance in Docker, an admin user, an API key wired into `mb`, your license, the databases you want to work with, and a git repository connected through Metabase's remote sync. Every step checks whether it is already done, so the wizard can be interrupted at any prompt and picked up again with the same command.
+
+After `init`, the rest is lifecycle: `rde start`, `rde stop`, `rde status`, `rde logs`, `rde open`, and `rde db add` when another database comes along. The instance lives under `~/.rde`, listens on `127.0.0.1` only, and is used through `mb` and through the agent skill.
+
+## Requirements
+
+- macOS or Linux
+- Node 20.19 or newer
+- git
+- Docker: Docker Desktop, OrbStack, or colima on a Mac; the `docker` package on Linux, with your user in the `docker` group
+
+`rde init` checks all four first and prints what is missing with the install hint for your platform.
+
+## Install
+
+```sh
+npm install -g @metabase/rde
+rde init
+```
+
+## What `rde init` does
+
+The wizard shows its checklist first (`✓` done, `○` to do, `–` not applicable), runs the steps still to do in order, and ends with a summary of your environment. A first run on a machine that has Docker but not yet `mb` starts like this:
+
+```
+┌  rde init
+✓ Check this machine
+○ Install the Metabase CLI
+○ Install the rde skill for your coding agents
+○ Install Metabase
+○ Start Metabase
+○ Create the admin user
+○ Connect the Metabase CLI
+○ Activate your license
+○ Connect your databases
+– Connect a git repository
+○ Summary
+│
+◆  Install @metabase/cli globally with npm?
+```
+
+The git step reads `–` until a license with remote sync is active; it is checked again when its turn comes, so a run that activates a license connects the repository in the same pass.
+
+| Step | What it does |
+| --- | --- |
+| Check this machine | Platform, Node version, git, and a reachable Docker daemon |
+| Install the Metabase CLI | `@metabase/cli` (`mb`) with the package manager that installed `rde` |
+| Install the rde skill | The `rde` skill from `metabase/agent-skills` for the agents you pick (Claude Code, Codex, Cursor, Gemini CLI, OpenCode, Windsurf) |
+| Install Metabase | Pulls `metabase/metabase-enterprise:latest` (or `--image`) and creates the container on port 13000 (or the first free port above it, or `--port`) |
+| Start Metabase | Starts the container and waits for `/api/health`, showing Metabase's own progress |
+| Create the admin user | Default credentials or your own, see below |
+| Connect the Metabase CLI | Creates an API key named `rde` and logs `mb` in with it under the profile `default`, so plain `mb …` uses the instance; if `default` already points at another Metabase, offers the named profile `rde` instead |
+| Activate your license | Optional. Paste a token, or press Enter to skip; remote sync needs one with the `remote-sync` feature |
+| Connect your databases | Paste a connection string (or fill the fields one by one), review, test, add; repeat |
+| Connect a git repository | Only with a license. Initializes or reuses a repository and points Metabase's remote sync at it |
+| Summary | URL, login, `mb` profile, databases, repository, and the command that opens each detected agent |
+
+The second run on the same machine detects every step as done and prints the summary only:
+
+```
+┌  rde init
+✓ Check this machine
+✓ Install the Metabase CLI
+✓ Install the rde skill for your coding agents
+✓ Install Metabase
+✓ Start Metabase
+✓ Create the admin user
+✓ Connect the Metabase CLI
+– Activate your license
+✓ Connect your databases
+– Connect a git repository
+○ Summary
+│
+◇  Your environment ────────────────────────────────────────────────────╮
+│                                                                       │
+│  Metabase: http://localhost:13000                                     │
+│  Login: admin@metabase.com / metabase123                              │
+│  Metabase CLI: mb <command>                                           │
+│  Databases:                                                           │
+│    Sample Database (sqlite, sync complete)                            │
+│    warehouse (postgres, sync complete)                                │
+│  Remote sync: not configured (no license)                             │
+│  rde skill: installed for Claude Code                                 │
+│  Start working:                                                       │
+│    claude  (Claude Code)                                              │
+│  Lifecycle: rde status, rde stop, rde start                           │
+│                                                                       │
+├───────────────────────────────────────────────────────────────────────╯
+│
+└  rde init finished
+```
+
+Ctrl-C at any prompt stops the run; `rde init` again continues from the first step that is not done. `rde init --from license` re-runs from a step on, `rde init --only databases` runs one step.
+
+## Credentials
+
+The admin-user step offers two variants:
+
+- **Use default credentials**: `admin@metabase.com` / `metabase123`. Both are written to `~/.rde` and shown in the summary and in `rde status`, so agents and scripts can log in without asking you. Pick this for a local instance nobody else can reach.
+- **Enter my own**: email and password are asked once, sent to Metabase, and never written anywhere by `rde`.
+
+In both variants `rde` creates an API key named `rde` in the Administrators group and stores it in the instance's state file with mode `0600`. That key is what `mb` and `rde` itself use afterwards.
+
+## What lands in `~/.rde`, and what never does
+
+```
+~/.rde/
+  config.json                        { schemaVersion, defaultInstance }
+  instances/default/
+    state.json                       mode 0600: runtime, url, admin credentials variant, API key, license
+                                     features, databases, git remote, step records
+    data/                            mounted into the container as /rde
+      metabase.db/metabase.db.mv.db  the H2 application database
+      plugins/                       driver jars (Oracle, Vertica) go here
+      content.git/                   the bare repository behind a local git remote
+```
+
+Three things are never written to disk or to a log by `rde`: the license token (it goes to Metabase, which keeps it in its own application database), a password you typed in the "Enter my own" variant, and a git access token for a hosted remote (Metabase encrypts it on its side). The `mb` profile `rde` lives where `mb` keeps its profiles, under `$XDG_CONFIG_HOME` or `~/.config`.
+
+The Metabase container publishes its port on `127.0.0.1` only, so the instance is reachable from this machine and from nothing else on the network.
+
+## Everyday commands
+
+| Command | What it does |
+| --- | --- |
+| `rde start` | Starts the instance and waits until it is healthy |
+| `rde stop` | Stops it; the data stays |
+| `rde status` | The init checklist, container state, health, databases with their sync state, remote sync state; `--json` for scripts |
+| `rde logs` | The container's log; `-f` keeps streaming |
+| `rde open` | Opens the instance in the browser |
+| `rde db add` | The same database wizard as in `init` |
+| `rde db list` | The connected databases; `--json` for scripts |
+| `rde doctor` | One table of checks with a fix hint per failure; `--fix` re-runs the init steps behind them; `--json` for scripts; exit 1 while anything fails |
+| `rde uninstall` | Removes the instance, its data, and the `mb` profile; see [Resetting](#resetting) |
+
+Everything else goes through the Metabase CLI, logged in as the profile `default`, so plain `mb …` talks to the instance:
+
+```sh
+mb db list
+mb git-sync status
+```
+
+If your `default` profile already pointed at another Metabase when `init` ran, it was left alone and the instance was connected as the profile `rde` (`init` asks; `--mb-profile default|rde` answers without a terminal). Then it is `mb --profile rde …`, or `export MB_PROFILE=rde` once per shell; the summary and `rde status` say which.
+
+## The git repository
+
+With a license that includes remote sync, the last step connects a git repository. You choose where the working directory lives (here, a new directory, or an existing repository) and where Metabase pushes:
+
+- **Local remote inside `~/.rde`** (default): a bare repository at `instances/default/data/content.git`, mounted into the container and reached as `file:///rde/content.git`. No account, no token. Your working directory gets it as `origin` (or as `rde-remote` when `origin` already points elsewhere).
+- **Hosted remote**: a GitHub, GitLab, or Bitbucket `https://` URL plus a token with read and write access. `rde` tests the connection through Metabase before saving anything.
+
+A repository that already holds Metabase content is imported on the spot. Afterwards:
+
+```sh
+cd <your working directory>
+mb git-sync status
+mb git-sync export -m "what changed"
+git pull   # after an export, to see the YAML files
+```
+
+## Running `init` without a terminal
+
+Every prompt can be answered ahead of time, by a flag or by a JSON answers file, so an agent or a CI job can run `rde init` unattended. When stdin is not a terminal, the first prompt nothing answers fails with exit code 2 and a message naming the step, the prompt, and the flag or answers key that would have answered it; nothing hangs. A text prompt with a default (image, port, display name) takes its default, as Enter would.
+
+### Flags
+
+```sh
+rde init --agents claude-code,codex --admin default --skip-license \
+         --database "postgres=postgres://user:pass@localhost:5432/app" \
+         --git-dir ~/analytics --git-remote local
+```
+
+| Flag | Answers |
+| --- | --- |
+| `--agents claude-code,codex` / `--skip-agents` | Which agents get the skill |
+| `--image`, `--port` | The Docker image and host port |
+| `--admin default\|prompted`, `--admin-email`, `--admin-password` | The admin user; an email or password implies `prompted` |
+| `--license-token` / `--skip-license` | The license |
+| `--database <engine>=<connection string>` (repeatable) / `--skip-databases` | The warehouses |
+| `--git-dir`, `--git-remote local\|hosted`, `--git-url`, `--git-token` / `--skip-git` | The content repository; a remote or URL without `--git-dir` means the current directory |
+| `--yes` | Every confirmation (installing `mb`, skipping databases) |
+
+A secret flag (`--admin-password`, `--license-token`, `--git-token`) takes `-` to read one line from stdin, the way `mb auth login` takes its key, or a literal `$VAR` (single-quoted, so the shell leaves it alone) to read the environment; either keeps the value out of your shell history and process listings. Only one flag can read stdin.
+
+```sh
+echo "$PASSWORD" | rde init --admin-email me@example.com --admin-password - …
+rde init --license-token '$RDE_LICENSE_TOKEN' …
+```
+
+### Answers file
+
+`rde init --answers <file>` reads a JSON file with one section per step; flags win over the file, key by key. A `$VAR` string is read from the environment, and an unset variable is an error rather than an empty answer.
+
+```json
+{
+  "install-mb": { "package-manager": "npm" },
+  "agents-skills": { "agents": ["claude-code"] },
+  "runtime": { "image": "metabase/metabase-enterprise:latest", "port": 13000 },
+  "admin-user": { "variant": "prompted", "email": "me@example.com", "password": "$RDE_ADMIN_PASSWORD" },
+  "license": { "token": "$RDE_LICENSE_TOKEN" },
+  "databases": [
+    { "engine": "postgres", "uri": "$RDE_DATABASE_1_URI" },
+    { "engine": "bigquery-cloud-sdk", "fields": { "project-id": "acme", "service-account-json": "/keys/acme.json" }, "name": "Warehouse" }
+  ],
+  "git-sync": { "location": "new", "path": "~/analytics", "remote": "local-bare" }
+}
+```
+
+`"license": { "token": null }` skips the license, `"databases": []` connects none, `"git-sync": { "location": "skip" }` leaves the repository for later. A database is pasted as a `uri` or entered with `fields`, keyed by the driver's field names as the admin form shows them; a file field takes a path. Anything a `fields` entry leaves out is asked in the terminal, or named in the error without one.
+
+### Recording a run
+
+`rde init --print-answers <file>` writes, once an interactive run finishes, the answers file that reproduces it. Secrets are written as references, never as values: `$RDE_ADMIN_PASSWORD`, `$RDE_LICENSE_TOKEN`, `$RDE_GIT_TOKEN`, `$RDE_DATABASE_<n>_URI` for a connection string carrying a password, and `$RDE_DATABASE_<n>_<FIELD>` for a secret field. Set those variables and run `rde init --answers <file>` on the next machine.
+
+## Resetting
+
+`rde uninstall` returns the machine to the state before `rde init`: it stops and removes the container, forgets the `mb` profile it logged in (`default`, or `rde`; never a profile pointing at another Metabase), and deletes `~/.rde/instances/default` along with the `config.json` entry. It shows the list first and asks once; `--yes` skips the question. Your working directory and its git history are never touched, nor are the `mb` CLI and Docker itself.
+
+```sh
+rde uninstall                  # container, mb profile, ~/.rde/instances/default
+rde uninstall --keep-data      # container and profile only; `rde init` reinstalls from the kept data
+rde uninstall --remove-skill   # also the rde skill from your coding agents
+rde uninstall --remove-image   # also the Docker image (otherwise it stays for the next init)
+```
+
+Running it twice is a no-op. It also works with no `~/.rde` at all, so a leftover `rde-metabase-default` container or `mb` profile from a half-finished run is cleaned up the same way. Anything it could not remove is listed at the end and the command exits with 1.
+
+## Troubleshooting
+
+Start with `rde doctor`. It checks the machine (platform, Node, git, Docker, `mb`), the instance (container, port, health, admin user, API key, `mb` profile, license, databases, remote sync), and the skill for each detected coding agent, one row each with ✓, ✗, or – and a hint under every ✗:
+
+```
+✓ container    rde-metabase-default stopped (metabase/metabase-enterprise:latest)
+✓ port         13000 is free
+✗ health       unreachable
+               → `rde start`
+– admin user   Metabase is not healthy
+```
+
+`rde doctor --fix` lists the init steps that repair the failed rows, asks once (`--yes` skips the question), re-runs them, and prints the table again. A failure with no automatic fix, such as a port held by another program, keeps its hint and the exit code stays 1 until it is resolved.
+
+**Docker is not running.** The first step prints the check that failed, with what Docker said and what to do, and exits with code 2:
+
+```
+◇  This machine ──────────────────────────────────────────────────────────────────────────────────╮
+│                                                                                                 │
+│  ✓ macOS                                                                                        │
+│  ✓ Node v22.13.1                                                                                │
+│  ✓ git version 2.50.1                                                                           │
+│  ✗ Docker: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker    │
+│    daemon running?                                                                              │
+│    Install Docker Desktop (https://www.docker.com/products/docker-desktop/), OrbStack, or       │
+│    colima, and start it                                                                         │
+│                                                                                                 │
+├─────────────────────────────────────────────────────────────────────────────────────────────────╯
+Fix the items above and run rde init again
+```
+
+Start Docker (or `colima start`) and run `rde init` again. `rde start` and `rde status` report the same condition on an existing instance.
+
+**The port is in use.** Port 13000 is the default because 3000 is usually taken by a Metabase checkout's dev server. When 13000 is busy too, the runtime step says so and offers the next free port: `Port 13000 is in use by another program; 13001 is free`. Accept it, type another, or pass `--port`. A container that exists with a different port or image is reported with the `docker rm -f` command that clears it.
+
+**The license token is rejected.** Activation ends with `Metabase reports the token as invalid` (or another status Metabase returned). Check the token, and check that the container can reach `token-check.metabase.com`: activation needs outbound network access. Run `rde init --from license` to try again; press Enter at the prompt to skip the license and carry on without remote sync.
+
+## Development
+
+```sh
+bun install
+bun run check                 # tsc, oxlint + boundary linter, oxfmt, unit tests
+bin/rde-dev init              # runs from source with RDE_HOME and the mb config under .dev-state/
+```
+
+The end-to-end lane runs the built binary against real Docker: a seeded Postgres 16 sidecar on `127.0.0.1:15432` (`RDE_E2E_PG_PORT` to change it) and a fresh instance under `.e2e-state/`, with a fake `HOME` so the skill installer never touches your agents. `bun run build` must precede `bun run test:e2e`.
+
+```sh
+bun run build && bun run e2e:up && bun run test:e2e
+bun run e2e:down
+```
+
+With `MB_PREMIUM_EMBEDDING_TOKEN` in the environment the lane also activates the license and connects a local git remote; without it that test is reported as skipped.