kampodra 0.8.1 → 0.8.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "kampodra",
3
- "version": "0.8.1",
3
+ "version": "0.8.2",
4
4
  "description": "Registry-free deployment CLI for single-VM container stacks over SSH (Podman + kamal-proxy)",
5
5
  "license": "AGPL-3.0",
6
6
  "type": "module",
@@ -8,10 +8,10 @@
8
8
  "kampodra": "bin/kampodra.js"
9
9
  },
10
10
  "optionalDependencies": {
11
- "@talha7k/darwin-arm64": "0.8.1",
12
- "@talha7k/darwin-x64": "0.8.1",
13
- "@talha7k/linux-arm64": "0.8.1",
14
- "@talha7k/linux-x64": "0.8.1"
11
+ "@talha7k/darwin-arm64": "0.8.2",
12
+ "@talha7k/darwin-x64": "0.8.2",
13
+ "@talha7k/linux-arm64": "0.8.2",
14
+ "@talha7k/linux-x64": "0.8.2"
15
15
  },
16
16
  "engines": {
17
17
  "node": ">=20"
@@ -0,0 +1,118 @@
1
+ ---
2
+ name: kampodra
3
+ description: "Operates the kampodra CLI, a kamal-alternative that deploys containers or binaries to Linux VMs over SSH with podman and kamal-proxy — no registry in the image path, zero-downtime by default. Use when deploying or rolling back a release, running the standby-update bluegreen flow (deploy to the idle color then flip), recovering an interrupted rolling deploy, reading deploy logs/status, pushing or diffing remote env files, generating env from a schema, managing instance profiles, listing or verifying backups, planning a restore, managing DNS records on OCI or GCP, staging golden images, running tenant database migrations, sampling VM metrics, or preparing and wiping VMs. Triggers: deploy, rollback, podman, kamal-proxy, SSH, env, metrics, backup, DNS, gcp, gcloud, oci, bluegreen, flip, migrate, VM, Alpine."
4
+ license: AGPL-3.0
5
+ compatibility: Requires podman, ssh, and git locally; the target VM needs a Linux userland (Alpine or Ubuntu 24.04) with podman and kamal-proxy (vm-prepare installs them). Cloud families authenticate through the provider CLI itself (oci or gcloud).
6
+ metadata:
7
+ author: kampodra maintainers
8
+ version: 0.8.0
9
+ ---
10
+
11
+ # kampodra
12
+
13
+ ## Overview
14
+
15
+ kampodra ships a git repository's app container straight to a Linux VM over SSH: `podman build` locally, `podman save | ssh podman load` to stream the image (no registry in the path), env push, init-aware restart, a served-sha health gate, a kamal-proxy re-point, then a public smoke check. It also owns the full instance lifecycle: rollback, deploy history and pruning, logs and shell, env management, metrics, OCI Object Storage backups, OCI DNS records, tenant database migrations, and VM prepare/wipe.
16
+
17
+ Every host-aware command resolves its target the same way (first match wins):
18
+
19
+ - profile selection: `--profile` flag > `KAMPODRA_PROFILE` env > the config's `defaultProfile`
20
+ - host: `--host` flag > profile `host` > `KAMPODRA_HOST`
21
+ - ssh key: `--ssh-key` flag > profile `sshKey` > `KAMPODRA_SSH_KEY` (empty = ssh agent / `~/.ssh/config`)
22
+
23
+ The project's shape (container name, env-file path, ports, bucket, ...) resolves through the ladder:
24
+
25
+ ```
26
+ flags > KAMPODRA_* env > profile "project" block > kampodra.json > built-in defaults
27
+ ```
28
+
29
+ `kampodra config print` renders the fully resolved effective project config with per-field provenance — always the first debugging stop when a value is not what you expect. Full flag-level detail for every command lives in [references/commands.md](references/commands.md).
30
+
31
+ ## Prerequisites
32
+
33
+ - Local: Go 1.27+ to build; `git`, `podman`, and `ssh` on `PATH`.
34
+ - Remote: a Linux VM reachable over key-based SSH (kampodra targets Alpine + OpenRC; systemd hosts are detected for restart/stop paths). `vm-prepare` bootstraps a bare Alpine host with the podman stack, OpenRC services, and kamal-proxy.
35
+ - Build once: `go build -o kampodra ./cmd/kampodra` (source build is the supported install path). Verify with `kampodra --version`.
36
+
37
+ ## Configure once, deploy forever
38
+
39
+ 1. Create a profile (per-instance, stored in `~/.kampodra/config.json`, dir 0700 / file 0600; ssh keys are stored as paths only — never secrets):
40
+
41
+ ```
42
+ kampodra config init --name prod --host root@203.0.113.10 --ssh-key ~/.ssh/id_ed25519 --proxy-host app.example.com --set-default
43
+ ```
44
+
45
+ The first profile auto-becomes the default. Hosts/keys live only in profiles; the committed repo file `kampodra.json` carries project naming only.
46
+
47
+ 2. (Recommended) Commit a `kampodra.json` at the repo root naming YOUR app. The built-in defaults are deliberately generic placeholders (`app`, `/etc/kampodra/env`, `app-backups`, …) — set your real naming in the manifest (19 optional camelCase fields: `container`, `shadowSuffix`, `envFile`, `dataDir`, `bucket`, `objectPrefix`, `healthPath`, `healthPaths`, `proxyHost`, `services`, `imagePrefix`, `port`, `network`, `shadowProbePort`, `deployedShaFile`, `envClearKeys`, `dockerfile`, `migrateScript`, plus a `unit.env` map rendering repo-owned -e flags into the app unit; an optional `"$schema"` key points at docs/kampodra.schema.json for editor autocomplete and is ignored). The parse is strict: malformed JSON or an unknown key FAILS CLOSED — a typo errors (naming the valid keys) instead of silently resolving to defaults. See the field table in [references/commands.md](references/commands.md).
48
+
49
+ 3. Push env (values are never printed; only KEY + fingerprint):
50
+
51
+ ```
52
+ kampodra env push --file ./ops/env.production # or generate locally: kampodra env from-schema
53
+ ```
54
+
55
+ 4. First bootstrap on a fresh VM: `kampodra vm-prepare --host root@203.0.113.10` (idempotent; the app container is intentionally NOT started — the first deploy provides image and env).
56
+
57
+ ## First deploy and verification
58
+
59
+ ```
60
+ kampodra deploy # build HEAD (clean-tree gate) and run the full pipeline
61
+ kampodra status # live health through the proxy + deployment count + VM state
62
+ kampodra deploy logs --lines 100 # tail the app container (--since 2h catches boot-first lines)
63
+ ```
64
+
65
+ Deploy stamps the git sha. In deploy mode the clean tree is gated, so what HEAD names is what ships. `kampodra deploy list` shows history (VM sha-tagged images merged with the local ledger `~/.kampodra/deployments.jsonl`); `kampodra metrics` snapshots load/memory/disk/containers; `deploy prune` reclaims disk from old sha-tagged images (always preview with `--dry-run`).
66
+
67
+ ## Rollback: a flag, not a subcommand
68
+
69
+ There is NO `rollback` subcommand. Rollback is the flag `deploy --rollback [<sha7>]`:
70
+
71
+ ```
72
+ kampodra deploy --rollback ccc3333 # explicit target
73
+ kampodra deploy --rollback # resolves from the VM's deployed-sha stamp
74
+ ```
75
+
76
+ Target resolution: the explicit sha, else the VM's deployed-sha stamp file, else it DIES — it never falls back to git HEAD (rolling back must not redeploy the very build you are rolling back from). Rollback skips the build/stream when the tag still exists on the VM and never rewrites the stamp.
77
+
78
+ ## Zero-downtime deploys and recovery
79
+
80
+ `kampodra deploy --rolling` boots the new version as a shadow container, health-gates it BEFORE any traffic can reach it, then double re-points kamal-proxy (to the shadow, then back to the new main). Failure paths keep the safer state — but if an interrupted run exits loudly mid-switch, the shadow is left SERVING on purpose. Finish it with:
81
+
82
+ ```
83
+ kampodra deploy converge [<sha7>] # retag → restart → health gate → re-point to main → remove shadow
84
+ ```
85
+
86
+ A leftover shadow must be converged before the next `--rolling` deploy. `--drain-timeout` (default 10s, rolling only) bounds the proxy-switch confirmation.
87
+
88
+ ## Safety: destructive and guarded operations
89
+
90
+ | Operation | Behavior | Guard |
91
+ |---|---|---|
92
+ | `vm-wipe` | Stops/disables services, removes project containers, prunes images, deletes env/stamp/state | `--yes` REQUIRED; `--keep-data` spares the data dir; `--force` overrides a services-mismatch refusal (wrong host?) |
93
+ | `backup restore-plan <object>` | PRINTS the documented recovery sequence | Never executes anything |
94
+ | `backup verify <file>` | Read-only: uploads to VM scratch files and runs the VM's `restore-verify` | Never writes into the data dir |
95
+ | `deploy prune` | Removes old sha-tagged images (keeps running + ts-rollback + newest N) | `--dry-run` prints exact commands first |
96
+ | `migrate` | Writes db schema | Stop-first guard: refuses while the api service runs unless `--allow-running` |
97
+ | `env push` | Overwrites the remote env file atomically | Preview first with `env diff <local-file>` (exit 1 = differs) |
98
+ | `vm-prepare` | Mutates a bare host (packages, sshd, services) | Idempotent; safe to re-run |
99
+
100
+ ## Gotchas
101
+
102
+ - Defaults are GENERIC placeholders (`app`, `/etc/kampodra/env`, `app-backups`, `/up`), not your app's names — commit a `kampodra.json` before the first deploy against a new tree, or check `config print`.
103
+ - `migrate` runs the app's own script on the VM: the path comes from the `migrateScript` config (default `scripts/migrate-db.ts`), executed with `pnpm exec tsx` inside `--repo-root` (or `KAMPODRA_REPO_ROOT`). Point `migrateScript` at your repo's real layout.
104
+ - `dns`/`backup`/`bluegreen` are multi-provider (oci | gcp behind the neutral dispatch; one `RunCloud` seam): they authenticate through the provider CLI's own auth, and kampodra passes no auth flags unless the instance profile's `cloud` block overrides it (oci: `profile`/`compartment`/`instancePrincipal`; gcp: `provider: gcp` + `project`/`account`); `OCI_PROFILE`/`OCI_COMPARTMENT` env work as the oci escape hatch, else the provider CLI resolves natively; gcp scopes natively via the block's `project` (dns also takes `dnsCompartment` + `--dns-compartment` for split-compartment zones). `image-import` is OCI-only by design (a `gcp` profile fails closed — Cloud Build trust-shape). `bluegreen provision` pins `VM.Standard.A1.Flex` on oci, `e2-standard-2` on gcp. `--profile` always means the kampodra INSTANCE profile.
105
+ - Fleet profiles: `--group web` filters `config list` / `status` / `deploy list`; `config clone prod prod-2 --host root@<ip>` stamps out a sibling with explicit copies (no live link).
106
+ - `bluegreen` manages the reserved-IP pair (`status | init | provision <color> | deploy --to <color> | flip --to <color> [--force] | rollback`) — pairs + reserved IP derive from LIVE provider state, no manual-IP fallback. `deploy --to <color>` is the standby-update composite: build once → push to the idle color → health-gate it there → flip (pipeline failure = no flip). A flip warns when the CURRENT holder is unhealthy (no healthy fallback after cutover). `image-import --image <qcow2>` stages+imports golden images on oci (warns on non-UEFI firmware). See [references/commands.md](references/commands.md).
107
+ - `deploy` health-gates and smokes EVERY path in the manifest's `healthPaths[]` (default: exactly [`healthPath`]) — pin the routes that must answer, not just the shallow one. Binary fast-pushes are sha256-fingerprinted and byte-verified on the VM before the swap; a stale rebuild (bytes identical to the previous push under a new sha) warns loudly.
108
+ - No command ever prints env values — only KEY + fingerprint tables (value length + first 2 chars). Raw values move only through `env push`'s upload and `env pull`'s stdout/`--out` payload.
109
+
110
+ ## Common flows
111
+
112
+ - Bad release, revert now: `kampodra deploy --rollback` (stamp-resolved), confirm with `kampodra status`.
113
+ - Disk pressure: `kampodra metrics --disk-threshold 90` → `kampodra deploy prune --dry-run` → apply.
114
+ - Env drift suspected: `kampodra env diff ./ops/env.production` → `kampodra env push --file ./ops/env.production` → `kampodra deploy restart`.
115
+ - One-off inspection on the VM: `kampodra ssh df -h` (argv passes through verbatim), or inside the container `kampodra deploy shell exec -- df -h`.
116
+ - Schema change: stop-first `kampodra migrate --repo-root /srv/app` (failures are collected per tenant db; exit 1 means do NOT restart the API on a half-migrated estate).
117
+
118
+ For every command's full flags, defaults, and exit codes, read [references/commands.md](references/commands.md).
@@ -0,0 +1,555 @@
1
+ # kampodra command reference
2
+
3
+ Per-command deep reference for AI agents operating the `kampodra` CLI. Every
4
+ flag name, type, and default below is taken from the Cobra definitions in the
5
+ CLI source (`internal/command/*.go`). When this file and `kampodra <command>
6
+ --help` disagree, re-run `--help` and trust the binary's live output.
7
+
8
+ **Contents**
9
+
10
+ - [Global resolution rules](#global-resolution-rules)
11
+ - [Environment variables](#environment-variables)
12
+ - [kampodra.json (project manifest)](#kampodrajson-project-manifest)
13
+ - [deploy](#deploy) (+ [converge](#deploy-converge-sha7) / [list](#deploy-list) / [prune](#deploy-prune) / [logs](#deploy-logs) / [restart](#deploy-restart) / [shell](#deploy-shell))
14
+ - [env](#env) (list / push / pull / fingerprint / diff / from-schema)
15
+ - [ssh](#ssh)
16
+ - [config](#config) (init / list / show / print / set-default / remove)
17
+ - [backup](#backup) (list / download / verify / restore-plan)
18
+ - [vm-prepare](#vm-prepare)
19
+ - [vm-wipe](#vm-wipe)
20
+ - [metrics](#metrics)
21
+ - [dns](#dns) (records / add / rm)
22
+ - [migrate](#migrate)
23
+ - [status](#status)
24
+ - [Exit codes](#exit-codes)
25
+ - [Agent gotchas](#agent-gotchas)
26
+
27
+ ## Global resolution rules
28
+
29
+ Every host-aware command resolves its target the same way (first match wins):
30
+
31
+ - profile: `--profile` flag > `KAMPODRA_PROFILE` env > the config's `defaultProfile`
32
+ - host: `--host` flag > profile `host` > `KAMPODRA_HOST`
33
+ - ssh key: `--ssh-key` flag > profile `sshKey` > `KAMPODRA_SSH_KEY` (empty = ssh agent / `~/.ssh/config`)
34
+ - proxy host: profile `project.proxyHost` block > `KAMPODRA_PROXY_HOST` > project default; the profile's legacy flat `proxyHost` field still wins when set
35
+
36
+ The project's shape (container name, env-file path, ports, bucket, ...) resolves
37
+ through the ladder:
38
+
39
+ ```
40
+ flags > KAMPODRA_* env > profile "project" block > kampodra.json > built-in defaults
41
+ ```
42
+
43
+ An unknown profile name fails closed naming the known set. Profiles live in
44
+ `~/.kampodra/config.json` (dir 0700 / file 0600); ssh keys are stored as paths
45
+ only — the config never contains secret material. `kampodra config print`
46
+ renders the fully resolved effective config with per-field provenance; make it
47
+ the first debugging stop. Every command supports `--help` (usage + examples).
48
+ ## Environment variables
49
+
50
+ | Variable | Effect |
51
+ |---|---|
52
+ | `KAMPODRA_HOST` | Fallback host (`user@ip` or ssh-config alias) when no flag/profile provides one |
53
+ | `KAMPODRA_SSH_KEY` | Fallback identity file path |
54
+ | `KAMPODRA_PROFILE` | Fallback profile name (beaten by `--profile`, beats `defaultProfile`) |
55
+ | `KAMPODRA_BUCKET` | The project ladder's canonical bucket override — beats the project config's `bucket` (only `--bucket` beats it) |
56
+ | `KAMPODRA_MIGRATE_SCRIPT` | Repo-relative migrate script path override for `migrate` |
57
+ | `KAMPODRA_REPO_ROOT` | Fallback VM repo checkout for `migrate` (`--repo-root` beats it) |
58
+ | `OCI_PROFILE` | OCI config profile override when the profile's `cloud` block sets none (else native resolution) |
59
+ | `OCI_COMPARTMENT` | REQUIRED by the oci legs of `dns`/`bluegreen`/`image-import` — no default; name or `ocid1.` ocid |
60
+ | `MIGRATE_JOBS` | Fallback parallelism for `migrate` (else 4; accepted range 1–64) |
61
+
62
+ Each `kampodra.json` field also has a `KAMPODRA_*` env override in the ladder:
63
+ `KAMPODRA_CONTAINER`, `KAMPODRA_SHADOW_SUFFIX`, `KAMPODRA_ENV_FILE`,
64
+ `KAMPODRA_DATA_DIR`, `KAMPODRA_BUCKET`, `KAMPODRA_OBJECT_PREFIX`,
65
+ `KAMPODRA_HEALTH_PATH`, `KAMPODRA_PROXY_HOST`, `KAMPODRA_SERVICES`,
66
+ `KAMPODRA_IMAGE_PREFIX`, `KAMPODRA_PORT`, `KAMPODRA_NETWORK`,
67
+ `KAMPODRA_SHADOW_PROBE_PORT`, `KAMPODRA_DEPLOYED_SHA_FILE`,
68
+ `KAMPODRA_ENV_CLEAR_KEYS`, `KAMPODRA_DOCKERFILE`.
69
+
70
+ ## kampodra.json (project manifest)
71
+
72
+ Repo-level manifest, committed per-project; discovered upward from the working
73
+ directory (nearest file wins). Carries project NAMING only — never hosts, ssh
74
+ keys, or secrets. All 19 fields optional, camelCase (plus the `unit` block), an optional `"$schema"`
75
+ key pointing at docs/kampodra.schema.json is accepted (ignored) for editor
76
+ autocomplete:
77
+
78
+ | Field | Type | Purpose |
79
+ |---|---|---|
80
+ | `container` | string | api container / service name |
81
+ | `shadowSuffix` | string | rolling-deploy shadow container suffix |
82
+ | `envFile` | string | remote env file path (profile blocks also accept the legacy `envFilePath` spelling; the manifest takes `envFile` only) |
83
+ | `dataDir` | string | tenant/app data dir on the VM |
84
+ | `bucket` | string | object-storage backup bucket |
85
+ | `objectPrefix` | string | backup object prefix |
86
+ | `healthPath` | string | app health endpoint behind the proxy |
87
+ | `proxyHost` | string | public TLS edge hostname |
88
+ | `services` | string[] | service roll call |
89
+ | `imagePrefix` | string | VM-local image reference prefix for deploy images |
90
+ | `port` | string | container's published/listening port |
91
+ | `network` | string | podman network shared with kamal-proxy |
92
+ | `shadowProbePort` | string | shadow container's loopback-only probe port |
93
+ | `deployedShaFile` | string | VM deployed-sha stamp (rollback's fallback) |
94
+ | `envClearKeys` | string[] | env keys owned by init — `env from-schema` drops them from varlock output |
95
+ | `dockerfile` | string | Containerfile/Dockerfile deploy builds |
96
+ | `migrateScript` | string | repo-relative db migrate script (`migrate` runs it on the VM) |
97
+ | `healthPaths` | string[] | EXTRA health-gate/smoke routes — the deploy probes EVERY path (default: exactly [`healthPath`]) |
98
+ | `unit` | object | repo-owned app-unit surface: `unit.env` renders each key as `-e K=V` on the app container's `podman run` (later-wins over renderer clear-env defaults; secrets stay in the env file) |
99
+
100
+ The parse is **strict fail-closed**: malformed JSON or an unknown key errors
101
+ instead of silently resolving to defaults. An empty object is valid. Built-in
102
+ defaults describe one sample app's shape, not neutral conventions — check
103
+ `config print` before the first run against a new tree.
104
+
105
+ ## deploy
106
+
107
+ Full pipeline: clean-tree gate → `podman build --build-arg GIT_SHA=<sha>` (sha
108
+ tag from `git rev-parse --short HEAD`) → `podman save | ssh podman load`
109
+ (registry-free stream) → env push (`--env-file`; 0600 + atomic mv, `API_GIT_SHA`
110
+ stamped) → init restart → VM-side served-sha health gate → kamal-proxy re-point
111
+ → public smoke → ledger append → disk report → keep-set image cleanup (running
112
+ + `ts-rollback` + newest 3 kept).
113
+
114
+ | Flag | Type | Default |
115
+ |---|---|---|
116
+ | `--host` | string | `""` (resolution ladder) |
117
+ | `--profile` | string | `""` (resolution ladder) |
118
+ | `--ssh-key` | string | `""` (agent / ssh config) |
119
+ | `--dockerfile` | string | `""` → project config's `dockerfile` (else `Dockerfile`) |
120
+ | `--sha <sha7>` | string | `""`; sha fragment required, never inferred |
121
+ | `--rollback [<sha7>]` | string | bare `--rollback` = stamp-resolved |
122
+ | `--rolling` | bool | `false` |
123
+ | `--drain-timeout` | int | `10` (seconds; requires `--rolling`) |
124
+ | `--disk-threshold` | string | `""` (percentage 0–100) |
125
+ | `--env-file` | string | `""` |
126
+ | `--skip-smoke` | bool | `false` |
127
+ | `--refresh-config` | bool | `false` — compatibility no-op (a warning prints) |
128
+
129
+ - `--sha` streams an EXISTING local build: no rebuild, clean-tree gate
130
+ skipped. `--rollback` and `--sha` are exclusive; `--rolling` and
131
+ `--rollback` are exclusive.
132
+ - `--disk-threshold` fails closed BEFORE the build when VM disk usage >= pct; an
133
+ unreadable df with the flag set is also an error. Disk > 90% always warns
134
+ loudly, flag or not.
135
+ - Deploy mode requires a git repo (the sha is stamped). Rollback target
136
+ resolution: explicit sha > the VM's deployed-sha stamp file > DIE — never git
137
+ HEAD. Rollback skips build/stream when the tag still exists on the VM, streams
138
+ from this machine when it does not, and never rewrites the stamp. A positional
139
+ in rollback mode is the sha (given twice = error); any other positional is
140
+ rejected.
141
+
142
+ Example: `kampodra deploy --host root@203.0.113.10 --rolling --env-file ./ops/env.production`
143
+
144
+ ### deploy converge [&lt;sha7&gt;]
145
+
146
+ Finishes an interrupted `--rolling` deploy: retag → restart → health gate →
147
+ re-point to main → remove the shadow. Flags: `--host`, `--profile`,
148
+ `--ssh-key`, `--drain-timeout` (int, default 10). The sha comes from the
149
+ argument (must be a sha fragment) or the shadow's own image — never HEAD. A
150
+ leftover shadow must be converged before the next `--rolling`.
151
+ Example: `kampodra deploy converge --host root@203.0.113.10`
152
+
153
+ ### deploy list
154
+
155
+ Deployment history: the VM's sha-tagged images (running one marked) merged with
156
+ the local ledger `~/.kampodra/deployments.jsonl`, plus a `Total deployments`
157
+ footer. Flags: `--host`, `--profile`, `--ssh-key`, `--all-profiles` (bool,
158
+ default false — one section per configured profile, sorted; errors when no
159
+ profiles exist), `--group <g>` (one section per profile in that group).
160
+ Same-tag rows merge to ONE row showing the NEWEST deploy timestamp. An
161
+ unreachable VM degrades to a ledger-only view with a warning, not an error.
162
+ Example: `kampodra deploy list --profile prod --all-profiles`
163
+
164
+ ### deploy prune
165
+
166
+ Reclaims VM disk by removing OLD sha-tagged deploy images. ALWAYS kept: the
167
+ currently-running image, `ts-rollback`, and the newest `--keep` N.
168
+ Flags: `--host`, `--profile`, `--ssh-key`, `--keep` (string, default `"2"`,
169
+ non-negative integer), `--dry-run` (bool, default false — prints the exact
170
+ `podman rmi` commands and removes nothing). Refuses to prune if the image list
171
+ fails over ssh. After applying, reports disk on `/var/lib/containers` and warns
172
+ above 90%.
173
+ Example: `kampodra deploy prune --host root@203.0.113.10 --dry-run`
174
+
175
+ ### deploy logs
176
+
177
+ Tails the running api container's logs. Flags: `--host`, `--profile`,
178
+ `--ssh-key`, `--lines` (string, default `"100"`, positive integer),
179
+ `--follow` (bool, default false — streams until ctrl-c with a clean exit; a
180
+ non-zero remote exit code propagates), `--since` (Go duration like `2h` or
181
+ RFC3339 — passed verbatim to `podman logs`; catches boot-first lines a bare
182
+ `--tail` cuts off).
183
+ Example: `kampodra deploy logs --host root@203.0.113.10 --lines 200 --since 30m --follow`
184
+
185
+ ### deploy restart
186
+
187
+ Restarts the api service, init-aware: `rc-service` on Alpine (supervise-daemon
188
+ re-runs `podman run` on `:latest`), `systemctl` on systemd hosts, then a
189
+ best-effort status check. Flags: `--host`, `--profile`, `--ssh-key`.
190
+ Example: `kampodra deploy restart --profile prod`
191
+
192
+ ### deploy shell
193
+
194
+ Interactive `sh` inside the running api container (forced tty), or a one-shot
195
+ via `exec -- <cmd>...`. The `--` separator is REQUIRED for one-shot mode (pflag
196
+ consumes it; the command vector is located via ArgsLenAtDash). Arguments are
197
+ POSIX sh-quoted before composition. Flags: `--host`, `--profile`, `--ssh-key`.
198
+ The container's exit code propagates.
199
+ Example: `kampodra deploy shell --host root@203.0.113.10 exec -- df -h`
200
+
201
+ ## env
202
+
203
+ The remote app env file (project config's `envFile` path, 0600). Values are
204
+ NEVER printed — every display is a KEY + fingerprint table (value length +
205
+ first 2 chars). Raw values move only through `push`'s upload and `pull`'s
206
+ stdout/`--out` payload. Subcommand flags: `--host`, `--profile`, `--ssh-key`
207
+ (standard ladder) except where noted.
208
+
209
+ - `env list` — KEY + fingerprint table of the remote file.
210
+ - `env push --file <f>` — `--file` string, required; refuses files with no
211
+ `KEY=VALUE` lines; uploads verbatim: remote 0600 from creation (umask 077) +
212
+ atomic `mv`; prints the fingerprint table and a restart hint (does NOT
213
+ restart itself).
214
+ - `env pull [--out <file>]` — `--out` string, default `""`. Without it the raw
215
+ payload goes to stdout and the masked summary to stderr; with it the file is
216
+ written 0600 and the summary goes to stdout.
217
+ - `env fingerprint [--file <f>]` — `--file` string, default stdin. Previews the
218
+ masking for a LOCAL file; no host needed.
219
+ - `env diff <local-file>` — exactly one positional arg. Fingerprint-level diff
220
+ of local vs remote; **exit 1 = differs** (GNU diff convention).
221
+ - `env from-schema [--repo <path>] [--schema <path>] [--out <file>]` —
222
+ kampodra-native. Generates the env file from the repo's committed
223
+ `.env.schema` via varlock (`node_modules/.bin/varlock` must exist in the
224
+ repo; `pass()` refs resolve through varlock + the pass store, values never
225
+ echo); `envClearKeys`-owned keys are dropped. Defaults: `--repo` `"."`
226
+ (resolved via `git rev-parse --show-toplevel`; no git repo = error),
227
+ `--schema` `"apps/api/.env.schema"`, `--out` `""` (= push through the push
228
+ flow instead of writing locally, 0600).
229
+
230
+ Example: `kampodra env diff ./ops/env.production --host root@203.0.113.10`
231
+
232
+ ## ssh
233
+
234
+ Host-level ssh passthrough (kampodra-native). Flags: `--host`, `--profile`,
235
+ `--ssh-key`. Flag parsing is NOT interspersed: everything after the first
236
+ non-flag argument belongs to the remote command verbatim. No command =
237
+ interactive login (forced tty). ssh's exit code propagates.
238
+ Example: `kampodra ssh --host root@203.0.113.10 df -h`
239
+
240
+ ## config
241
+
242
+ Per-instance profiles in `~/.kampodra/config.json` (dir 0700 / file 0600).
243
+ FLAT — no inheritance; `group` is a cosmetic list filter. Persistent flags:
244
+ `--name` (string), `--host` (string), `--ssh-key` (string, path only, tilde
245
+ expanded), `--proxy-host` (string), `--group` (string), `--profile` (string),
246
+ `--set-default` (bool, false), `--force` (bool, false).
247
+
248
+ - `config init` — create or `--force`-refresh a profile. `--name` required:
249
+ `[A-Za-z0-9][A-Za-z0-9_-]*`, max 64. The first profile auto-becomes the
250
+ default. Re-init refreshes flat fields, keeps the tooling-owned cached init
251
+ verdict unless host/ssh-key changed, and always preserves the profile's
252
+ `project` block (config init never manages it).
253
+ - `config list [--group <g>]` — all profiles, default marked `*`.
254
+ - `config show [--profile <name>]` — one profile's fields (else the default).
255
+ - `config print [--profile <name>]` — the fully resolved effective project
256
+ config with per-field provenance (flag/env/profile/repo-file/default). The
257
+ debugging tool for the resolution ladder.
258
+ - `config set-default <name>` — repoint `defaultProfile` (max 1 positional).
259
+ - `config remove <name> [--force]` — removing the LAST profile requires
260
+ `--force`; if the default was removed it reassigns sorted-first.
261
+
262
+ Example: `kampodra config init --name prod --host root@203.0.113.10 --ssh-key ~/.ssh/id_ed25519 --proxy-host app.example.com --set-default`
263
+
264
+ ## backup
265
+
266
+ OCI Object Storage backups via the `oci` CLI's own auth — kampodra passes
267
+ no auth flags unless the profile's `cloud` block overrides it, else the
268
+ oci CLI resolves natively; kampodra never accepts, stores, or logs
269
+ credential material. Persistent flags:
270
+
271
+ | Flag | Type | Default | Meaning |
272
+ |---|---|---|---|
273
+ | `--bucket` | string | `""` | bucket; ladder: `--bucket` > `KAMPODRA_BUCKET` > project config `bucket` |
274
+ | `--prefix` | string | `""` | object name prefix for `list` |
275
+ | `--out` | string | `""` | `download` destination (else the object's basename) |
276
+
277
+
278
+ | `--host` | string | `""` | VM target for `verify` |
279
+ | `--ssh-key` | string | `""` | identity file for `verify` |
280
+ | `--migrated-topology` | bool | `false` | `verify`: pass restore-verify's drill flag through |
281
+
282
+ - `backup list [--prefix <p>]` — objects in the bucket. LIST-denied by policy:
283
+ prints the exact policy shape needed on stderr, **exit 1** (GET works without
284
+ LIST once the object name is known).
285
+ - `backup download <object> [--out <file>]` — pre-creates the file 0600, fetches
286
+ via `oci get`, then verifies sha256/md5 against the object's digest metadata;
287
+ mismatch prints FAIL and **exits 1**; no metadata = integrity noted as skipped.
288
+ - `backup verify <local.db|.tgz>` — read-only drill. Accepts `.db`/`.sqlite`/`.sqlite3`
289
+ or `.tgz`/`.tar.gz`. Needs a VM target (`--host` / the legacy instance-profile
290
+ flag above / ladder). Each db is uploaded to a VM scratch file
291
+ `/tmp/kampodra-verify-<n>-<crypto-random-hex>.db` (created under umask 077,
292
+ ALWAYS removed — cleanup never masks the verdict), then the VM's
293
+ `/usr/local/bin/restore-verify` runs as `restore-verify -db <path>`.
294
+ `--migrated-topology` appends restore-verify's `--migrated-topology` drill
295
+ flag (bulk-migration chain shape downgraded to warnings). For a `.tgz`, the
296
+ archive is extracted to a local temp dir (path-traversal-refusing) and every
297
+ `.db`/`.sqlite` member is verified sorted; `root/root.db` members are
298
+ SKIPPED — the root db is schema-only, not in restore-verify scope. Never
299
+ writes into the data dir. Any member failure fails the command.
300
+ - `backup restore-plan <object> [--bucket <name>]` — PRINTS the documented
301
+ recovery sequence (stage → scp to scratch → verify → stop api → snapshot →
302
+ swap in → start + smoke). Never executes anything: no oci call, no ssh, no
303
+ mutation.
304
+
305
+ Example: `kampodra backup verify /tmp/restore.db --host root@203.0.113.10`
306
+
307
+ ## vm-prepare
308
+
309
+ First-run bootstrap of a bare Alpine VM; idempotent, safe to re-run. Sequence:
310
+ sanity gates (UEFI boot, no systemd, OpenRC tooling) → sshd hardening (drop-in +
311
+ Include + restart, then gated) → apk community repo + podman stack (netavark) →
312
+ project state dir + `/etc/containers/registries.conf` → sysctl
313
+ `net.ipv4.ip_unprivileged_port_start=80` → OpenRC services (api container +
314
+ kamal-proxy via supervise-daemon, rc-update'd into the default runlevel) →
315
+ `kampodra-anchor` watcher (inert without its conf) → kamal-proxy image pulled +
316
+ service UP. The app container is intentionally NOT started on a fresh VM — the
317
+ first deploy provides image and env (a converge runs only when a previous
318
+ deploy left both). The first TLS certificate issues during the first deploy.
319
+ Readiness loops: ssh 60 × 5s, proxy container 20 × 3s. Accepts unknown host
320
+ keys (first contact).
321
+
322
+ | Flag | Type | Default |
323
+ |---|---|---|
324
+ | `--host` | string | `""` (required in practice for a new VM) |
325
+ | `--profile` | string | `""` |
326
+ | `--ssh-key` | string | `""` |
327
+
328
+ | `--ansible <playbook>` | string | `""` — run this ansible-playbook against the host AFTER bootstrap (temp inventory file, `--private-key` from the resolved key; failure is fatal) |
329
+
330
+ Example: `kampodra vm-prepare --host root@203.0.113.10`
331
+
332
+ ## vm-wipe
333
+
334
+ kampodra-native teardown of the project stack on the target VM: stop + disable
335
+ every project service, remove the project containers (app, its rolling-deploy
336
+ shadow, kamal-proxy), `podman image prune -a -f`, delete the env file,
337
+ deployed-sha stamp, anchor conf, and state dir; the tenant data dir goes LAST.
338
+ DESTRUCTIVE — **`--yes` is REQUIRED**; without it nothing runs.
339
+
340
+ | Flag | Type | Default |
341
+ |---|---|---|
342
+ | `--yes` | bool | `false` — confirm the teardown (required) |
343
+ | `--keep-data` | bool | `false` — preserve the tenant data dir |
344
+ | `--force` | bool | `false` — override the services-mismatch refusal |
345
+ | `--host` | string | `""` |
346
+ | `--profile` | string | `""` |
347
+ | `--ssh-key` | string | `""` |
348
+
349
+ Match check: if NO project container is present AND no project service is
350
+ running, it refuses (`--force` to mean it — wrong host?). Individual step
351
+ failures are tolerated and reported, not fatal.
352
+ Example: `kampodra vm-wipe --profile prod --yes --keep-data`
353
+
354
+ ## metrics
355
+
356
+ One-shot VM snapshot over SSH (load, memory, disk breakdown, containers, top
357
+ procs) — one round-trip per snapshot, busybox-safe remote commands. NOT
358
+ continuous monitoring.
359
+
360
+ | Flag | Type | Default |
361
+ |---|---|---|
362
+ | `--host` | string | `""` |
363
+ | `--profile` | string | `""` |
364
+ | `--ssh-key` | string | `""` |
365
+ | `--disk-threshold` | string | `"90"` (1–100) — **exit 1** when root disk usage >= this threshold |
366
+ | `--watch` | string | `""` — re-snapshot every N seconds (positive int; ctrl-c ends, exit 0) |
367
+ | `--count` | string | `""` — bound the snapshot count (positive int) |
368
+
369
+ Above 90% root disk always warns (exit 0); only the `--disk-threshold` threshold
370
+ fails. Requires a resolved target.
371
+ Example: `kampodra metrics --host root@203.0.113.10 --disk-threshold 85`
372
+
373
+ ## dns
374
+
375
+ DNS records via the provider CLI's own auth ONLY (oci | gcp) — no auth
376
+ flags unless the profile's `cloud` block overrides (oci:
377
+ `profile`/`compartment`/`instancePrincipal`; gcp: `provider: "gcp"` +
378
+ `project`/`account`, else gcloud's active config); never credential
379
+ material. An OCI compartment is REQUIRED for oci only (block or
380
+ `OCI_COMPARTMENT` env), no default (a plain name resolves through
381
+ `iam compartment list`; an `ocid1.`
382
+ value is used as-is); gcp scopes via the block's `project`. Zone
383
+ resolution: `--zone` (ocid → name fetched; name →
384
+ as-is, trailing dot trimmed) else the compartment must hold EXACTLY one zone —
385
+ several → fail listing them (**exit 1**), zero → error. Record types are
386
+ pinned to `A | AAAA | CNAME`; all validation runs locally before any
387
+ provider call.
388
+
389
+ | Flag (persistent) | Type | Default |
390
+ |---|---|---|
391
+ | `--zone` | string | `""` |
392
+ | `--name` | string | `""` — DNS label or fqdn inside the zone |
393
+ | `--type` | string | `""` — A, AAAA, or CNAME |
394
+ | `--value` | string | `""` — type-shaped target |
395
+ | `--ttl` | string | `"300"` (60..172800) |
396
+ | `--profile` | string | `""` — instance profile (carries the `cloud` block) |
397
+ | `--dns-compartment` | string | `""` — dns-only scope for split-compartment tenancies: flag > cloud block `dnsCompartment` > `compartment`; the no-zones error names the compartment it searched |
398
+
399
+
400
+ - `dns records [--zone <id-or-name>]` — list records: domain / type / ttl / value.
401
+ - `dns add --name <label> --type <t> --value <v> [--ttl <s>]` — merges into the
402
+ existing RRSet (round-robin records survive); already present = no-op.
403
+ - `dns rm --name <label> --type <t> --value <v>` — filters the value out of the
404
+ RRSet (an emptied RRSet is removed entirely); no matching record = error.
405
+
406
+ Example: `kampodra dns add --name app --type A --value 203.0.113.10 --zone example.com`
407
+
408
+ ## migrate
409
+
410
+ Tenant db migrations over SSH, applied THROUGH the repo checkout ON the VM
411
+ (the app's own script, path from the `migrateScript` config — default
412
+ `scripts/migrate-db.ts`; no parallel implementation; runs via `pnpm exec
413
+ tsx`). Order: root.db first
414
+ (`root/root.db` preferred, legacy flat `root.db` honored), then `tenant_*.db`
415
+ sorted, bounded-parallel. Preconditions are probed in one round-trip and fail
416
+ closed: repo root (must contain the migrate script), pnpm, tenant data dir.
417
+
418
+ | Flag | Type | Default |
419
+ |---|---|---|
420
+ | `--repo-root` | string | `""` — the repo checkout ON THE VM; required (beats `KAMPODRA_REPO_ROOT`) |
421
+ | `--jobs` | string | `""` → `MIGRATE_JOBS` env, else `4` (1–64) |
422
+ | `--allow-running` | bool | `false` — migrate while the api service runs (SQLITE_BUSY risk) |
423
+ | `--host` | string | `""` |
424
+ | `--profile` | string | `""` |
425
+ | `--ssh-key` | string | `""` |
426
+
427
+ Stop-first guard: refuses while the api service is running (init-aware:
428
+ `rc-service` on OpenRC, `systemctl` on systemd; neither found = loud warning and
429
+ proceed). `--allow-running` overrides with a warning. Per-file failures are
430
+ COLLECTED (one bad tenant must not block the others); **exit 1 if any failed —
431
+ do NOT start the API on a half-migrated estate**.
432
+ Example: `kampodra migrate --host root@203.0.113.10 --repo-root /srv/app --jobs 1`
433
+
434
+ ## status
435
+
436
+ Live view: deployment count (ledger), health through the proxy
437
+ (`https://<proxyHost><healthPath>` + build-id), and — when a host resolves —
438
+ VM disk on `/var/lib/containers` (warn above 90%), sha-tagged image count +
439
+ prune estimate (keep 2), an init-aware service roll call, and with `--verbose`
440
+ the full metrics snapshot. Never fails on an unhealthy system — it renders
441
+ state.
442
+
443
+ | Flag | Type | Default |
444
+ |---|---|---|
445
+ | `--host` | string | `""` |
446
+ | `--profile` | string | `""` |
447
+ | `--ssh-key` | string | `""` |
448
+ | `--verbose` | bool | `false` (metrics snapshot; needs a target) |
449
+ | `--all-profiles` | bool | `false` — one full section per configured profile |
450
+
451
+ Example: `kampodra status --host root@203.0.113.10 --verbose`
452
+
453
+ ## bluegreen
454
+
455
+ Reserved-IP pair for zero-downtime cutovers, multi-provider (oci | gcp
456
+ behind the neutral dispatch). One instance can start alone; the pair
457
+ shares one RESERVED public IP (derived from LIVE provider state — no
458
+ local state file). Instances are `<prefix>-blue/green`. Auth: profile
459
+ `cloud` block / env / native (no cloud flags). Provision pins
460
+ `VM.Standard.A1.Flex` 2/12 on oci, `e2-standard-2` on gcp; the gcp
461
+ anchor is the reserved external IP itself (the guest watcher's metadata
462
+ poll is the documented follow-up).
463
+
464
+ - `deploy --to <blue|green> [--force-flip]` — the standby-update
465
+ composite: resolve the RUNNING idle color → build once (clean-tree
466
+ gate) → push + health-gate it on ITS own IP → flip (health-gated).
467
+ Pipeline failure = no flip (the pair keeps serving the holder). The
468
+ PRE-flip public smoke is skipped by design (the edge is still the
469
+ holder's); the flip's own ACME + served-sha verify through the
470
+ reserved IP is the post-cutover smoke.
471
+
472
+ - `status` — pair view: reserved IP + holder, both instances (ocid, AD),
473
+ per-color app health. Read-only.
474
+ - `init` — create the DORMANT reserved IP (`kampodra-active`); idempotent.
475
+ - `provision <blue|green>` — launch the sibling from the other color's AD
476
+ + subnet (`VM.Standard.A1.Flex` 2/12). `--os` (`alpine`\|`ubuntu`,
477
+ default `alpine`; ubuntu = 24.04 LTS) picks the golden-image prefix.
478
+ Native: newest `<container>-alpine*` (default) or
479
+ `<container>-ubuntu-24.04*` (`--os ubuntu`) custom image with UEFI_64
480
+ firmware. Else inject: platform-image launch
481
+ (`--platform-key` file / `OPS_SSH_PUBKEY`, `--platform-user` /
482
+ `PLATFORM_SSH_USER` default `ubuntu`) + golden qcow2 streamed onto the
483
+ boot disk (`--qcow2` / `ALPINE_QCOW2`; qemu-img required) + reboot +
484
+ per-OS boot verify (`/etc/alpine-release` 3.x for alpine, `ID=ubuntu`
485
+ in `/etc/os-release` for ubuntu). `--image-id` forces one image. Ends
486
+ with `vm-prepare` next steps (does not run it).
487
+ - `flip --to <color> [--force]` — ACME-first: health gate on the target's
488
+ own IP (unless `--force`) → anchor conf on the target guest (watcher
489
+ configures it) → reserved IP assigned (waits ASSIGNED) → ACME on the
490
+ target for the `<ip-dashes>.sslip.io` hostname → valid-cert HTTPS +
491
+ served-sha verify through the reserved IP → FLIPPED, then prints the
492
+ post-flip proxyHost guidance (the pre-flip host is DE-REGISTERED —
493
+ update the profile's `project.proxyHost` AND kampodra.json). Warns when
494
+ the CURRENT holder is unhealthy: after the cutover there is NO healthy
495
+ fallback (warn-only — rescuing a dead pair is legitimate). Post-assign
496
+ failure auto-rolls back to the other color's EXISTING anchor
497
+ (lookup-only) or to dormant, then cleans the failed target's guest.
498
+ Bad `--to` = **exit 2**.
499
+ - `rollback` — unassign to DORMANT: holder resolved by anchor match
500
+ (lookup-only), guest cleaned (conf + address), unassign waits AVAILABLE.
501
+
502
+ ## image-import
503
+
504
+ Upload a packer-built qcow2 and import it as an OCI custom image (build
505
+ machine; PARAVIRTUALIZED — alpine imports self-supported, ubuntu as
506
+ `Canonical Ubuntu` / `Ubuntu 24.04` via `--os alpine|ubuntu`, default
507
+ `alpine`). `--image` required;
508
+ `--bucket` (`KAMPODRA_IMPORT_BUCKET`, else `<container>-image-import`);
509
+ `--name-prefix` (default `<container>-alpine`, or
510
+ `<container>-ubuntu-24.04` with `--os ubuntu` — provision looks this up);
511
+ `--compartment`; `--keep-object`. Polls AVAILABLE (30m cap, terminal
512
+ states fail); non-UEFI firmware warns loudly (A1 rejects, x86 launches);
513
+ staged object deleted unless kept; prints the image OCID.
514
+
515
+ ## Exit codes
516
+
517
+ - `0` success (including degraded-but-loud views like `deploy list` without a VM).
518
+ - `2` unknown top-level command (the offender is named, the index reprinted).
519
+ - `1` everything else that errors, plus the defined semantic failures:
520
+ `env diff` differing (GNU diff convention), `metrics --disk-threshold` threshold
521
+ reached, `migrate` per-file failures (do-not-restart signal), `backup list`
522
+ LIST-denied, `backup download` digest mismatch, `dns` ambiguous multi-zone
523
+ compartment, `deploy logs --follow` / `ssh` / `deploy shell` propagating a
524
+ non-zero remote exit code.
525
+
526
+ ## Agent gotchas
527
+
528
+ - **Rollback is a FLAG, not a subcommand**: `kampodra deploy --rollback [<sha7>]`.
529
+ Bare `--rollback` resolves from the VM's deployed-sha stamp and DIES without
530
+ one — it never falls back to git HEAD.
531
+ - **Cloud auth is provider-native**: `backup`/`dns` (multi-provider: oci |
532
+ gcp) take no auth flags. The instance profile's `cloud` block (oci:
533
+ `profile`/`compartment`/`instancePrincipal`; gcp: `provider: "gcp"` +
534
+ `project`/`account`) overrides; `OCI_PROFILE`/`OCI_COMPARTMENT` env are
535
+ the oci escape hatch; with neither set the provider CLI resolves its
536
+ default itself. `bluegreen`/`image-import` are OCI-only today — a `gcp`
537
+ profile fails closed naming the surface.
538
+ `--profile` always means the kampodra INSTANCE profile.
539
+ - **`migrate` runs the app's own script**: path from the `migrateScript`
540
+ config (default `scripts/migrate-db.ts`) inside the VM repo checkout,
541
+ executed with `pnpm exec tsx`; `--repo-root`/`KAMPODRA_REPO_ROOT` is
542
+ mandatory.
543
+ - **`vm-wipe` is destructive**: `--yes` gates everything; the host-mismatch
544
+ refusal needs `--force`; `--keep-data` spares only the tenant data dir.
545
+ - `env push`/`deploy --env-file` overwrite the remote env atomically but do NOT
546
+ restart — follow with `deploy restart` or a deploy.
547
+ - `env pull` without `--out` puts RAW VALUES on stdout (summary on stderr);
548
+ never pipe it to a log.
549
+ - The manifest key is `envFile`; the profile `project` block's legacy key is
550
+ `envFilePath`. Only `envFile` is valid in `kampodra.json` (strict parse).
551
+ - The bucket ladder is `--bucket` > `KAMPODRA_BUCKET` (the project ladder's
552
+ own bucket layer) > project config `bucket`.
553
+ - `deploy prune` and `vm-wipe` both delete permanently — always `--dry-run` /
554
+ `--keep-data` first when unsure.
555
+ - `bluegreen` (reserved-IP pair: `status | init | provision | flip --to | rollback`) and `image-import --image` are live; flips are ACME-first with auto-rollback, imports warn on non-UEFI firmware.