@simon-says-build/harbormaster 0.6.0

package/DESIGN.md

@@ -0,0 +1,146 @@
+# harbormaster — design (v2: caller-owned sandboxes)
+
+Status: **live on `main`.** Caller-owned sandboxes are the shipped model. Validated
+end-to-end: acquire/exec boot + owner-death reap + PID-reuse guard, and concurrent
+same-project suites coexist with isolated data *and* websocket ports (the 9150-collision
+fix). Consumers (impulse + walk2gether CI/test lanes) run against it.
+
+## The problem
+
+One machine runs emulators for an open-ended set of Firebase projects — dev, CI, and
+the agent-qa runner spinning up *customer* apps on demand. Two things kept biting:
+
+1. **Port collisions.** Multiple processes booting `firebase emulators:start` on the
+   same ports (a shared per-project port, or the default 8080/4400) fight and kill each
+   other. `pkill` "fixes" it by nuking *everyone's* emulators — the exact antipattern a
+   coordinator must eliminate.
+2. **Data collisions.** Two processes sharing one project's emulator share its data —
+   e.g. a `clearFirestoreData` in one wipes the other, or docs accumulate across test
+   cases. Concurrent callers of "the impulse emulator" must NOT share an instance.
+
+## The model: caller-owned sandboxes
+
+There is **one primitive** — a caller asks for a sandbox and the daemon gives it a
+private one, reclaimed when the caller exits:
+
+```
+acquire { cwd, owner_pid, owner_start, only? }  ->  { leaseId, ports, projectId }
+```
+
+- **Issuance is unique per caller, not per project.** Two processes that both "want an
+  impulse emulator" get two independent suites (own block, own fresh data). No sharing,
+  so no collision.
+- **The lease is owned by the caller's process.** The daemon reaps the sandbox — kills
+  the emulators, frees the block — when `owner_pid` dies. The caller's death *is* the
+  release; there is nothing to remember to release, and never a reason to `pkill`.
+- **No registry.** A "project" is just the `firebase.json` in `cwd`. `project_id` comes
+  from `.firebaserc`. Ports come from a pool block. Nothing is hand-authored.
+- **`firebase.json` is portless** — the daemon injects the block's ports into a temp
+  config (`.firebase.brokered-*.json`, git-ignored). A bare `firebase emulators:start`
+  is intentionally unsupported; everything goes through the daemon.
+- **Discovery is the `acquire` response.** The owner gets its ports back and passes them
+  to its app/tests via env (`acquire --env` prints `export FIREBASE_*_EMULATOR_PORT` +
+  `GCLOUD_PROJECT`). There is no shared "project ports" lookup, because there is no
+  shared instance.
+
+## Three flavors
+
+- **`acquire` — the daemon boots the suite.** For callers that just want a working
+  backend (a test process, an interactive dev shell). Returns the ports.
+- **`allocate` — the daemon reserves a block; the caller boots its own suite.** For the
+  agent-qa run driver (`envs/_lib/backends/firebase-emulators.sh`), which has bespoke
+  build/wipe/seed logic. Returns a base; the driver derives ports by the block offsets.
+- **`allocate --spawn CMD` — the daemon RUNS the lease's process.** For long-lived
+  dev-env processes (Metro for e2e) that must outlive the calling shell — in
+  particular agent sessions, whose harness kills any process the agent backgrounds.
+  The lease is ownerless (daemon-owned): liveness is the spawned pid (zombie-aware —
+  the daemon `waitpid`s its own children, since a zombie passes `kill -0`), the TTL is
+  the backstop, and `release <id>` is the off switch. Spawn output goes to the lease
+  log in the state dir.
+
+`acquire` and plain `allocate` carry an `owner_pid` and are reaped the same way;
+`--spawn`/`--detach` leases default to no owner, because their *immediate* parent
+exits within seconds of the call and would reap the lease with it.
+
+That default proved too coarse. "Daemonised" decides who reaps the child; it does
+not mean nobody is interested. `--owner auto` walks up from the CLI past the
+transient shell/npm wrappers and records the first ancestor that outlives the
+call — under an agent harness, the agent process — so the lease dies with that
+session instead of lingering for its full multi-day TTL. It deliberately returns
+*no* owner when the walk reaches a boundary like an editor or a login session,
+rather than pretending a process that lives for days is a meaningful owner.
+
+## Daemon version handover
+
+Any copy of the CLI (a global install, a repo checkout, some project's
+`node_modules`) may be the first to start the daemon, so the daemon on port 4999 can
+be arbitrarily stale. `/health` reports the daemon version; `_ensure` in a NEWER
+client POSTs `/shutdown` to an older daemon and starts its own copy. This is safe by
+construction: leases live in the ledger on disk, and every leased process (suites,
+spawns) runs in its own session via `start_new_session`, so the successor daemon
+reloads the ledger and carries on — the handover tears nothing down. Older clients
+never downgrade a newer daemon.
+
+## Ownership & teardown (coordinated only — never `pkill`)
+
+A lease is reclaimed when ANY of:
+
+1. **Owner death** — `owner_pid` is gone. This is the primary, expected path.
+2. **PID reuse guard** — "alive" means *same pid AND same start time* (`ps -o lstart`),
+   so a recycled PID number doesn't keep a dead owner's sandbox alive.
+3. **Working directory gone** — the lease's `cwd` no longer exists. A deleted
+   worktree can never need its Metro again, and this is the one signal that is
+   unambiguous for daemon-spawned leases. Only a *confirmed* absence counts: if
+   the parent directory is missing too, that reads as an unmounted volume rather
+   than a deleted project, and the lease is left alone.
+4. **TTL** — a backstop for a caller that never dies (misconfigured / detached).
+5. **Emulator liveness** — dead suite process → reaped immediately; ports down (with
+   the process alive) only reap after a grace window of continuous downtime, for both
+   daemon-booted suites and reserved (caller-booted) blocks. A single failed connect
+   check on a loaded machine must never kill a healthy suite.
+
+Teardown always `killpg`s the whole process group so the emulator's **JVM children are
+reaped** — a v1 bug leaked Firestore/Storage JVMs because only the node parent was
+killed (verify this in v2). The daemon owns lifecycle end-to-end; callers use
+`acquire`/`release`, never `pkill`.
+
+## Owning PID is the caller, not the CLI
+
+`harbormaster acquire` is ephemeral — it exits the moment it has the response. So the
+CLI defaults `owner_pid` to **its parent** (`os.getppid()` — the calling shell or test
+runner), or the caller passes `--owner PID` explicitly (`$$` in a shell, `process.pid`
+in node). The sandbox lives as long as that owner does.
+
+## API
+
+```
+POST /acquire   {cwd, owner_pid, owner_start, only?, ttl?}  -> {leaseId, ports, projectId}
+POST /allocate  {owner_pid, owner_start, ttl?, label?}      -> {leaseId, base}
+POST /release   {leaseId}                                   -> {released}
+POST /heartbeat {leaseId, ttl?}                             -> {ok}          # extend TTL
+GET  /status                                               -> {leases:[...]}
+GET  /health
+POST /shutdown
+```
+
+CLI: `acquire` / `up` (alias) / `down` (release your lease) / `release <id>` / `status`
+/ `leases` / `allocate` / `serve`. Pool: blocks of 20 ports from 11000–14999, offsets
+`auth=+0 firestore=+1 functions=+2 storage=+3 ui=+4 hub=+5 logging=+6 pubsub=+7
+eventarc=+8 database=+9` (kept in sync with the run driver's derivation).
+
+## What changes from v1
+
+- Drop the registry (`~/.config/harbormaster.json`) and `list`/`apply`/`check`.
+- Drop sticky-per-project (`up`/`ports` as a shared instance). Everything is a
+  caller-owned lease.
+- `firebase.json` everywhere is portless (already done for agent-qa/impulse/walk2gether/
+  tuturno on their branches).
+- Consumers (`app.config`, scrub scripts, CLAUDE.md tables) stop hard-coding ports and
+  instead `eval "$(harbormaster acquire --cwd . --env)"` (owner = their process).
+
+## Open items before this goes live
+
+- Validate `acquire` boots + owner-death reap + PID-reuse guard on a quiet machine.
+- Confirm `killpg` reaps JVM children (fix the v1 leak if not).
+- Migrate the agent-qa run driver to pass `owner_pid` to `/allocate`.
+- Migrate the app.config / scrub-script / CLAUDE.md consumers to `acquire --env`.

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Michael Dawson
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,109 @@
+# harbormaster
+
+> Formerly `firebase-ports`. The scope outgrew the name: leases now cover Metro/Expo
+> ports and private iOS simulators too, not just Firebase emulators. The npm package
+> still installs a `firebase-ports` bin alias, and the daemon state dir is unchanged,
+> so existing consumers keep working.
+
+A localhost daemon that leases each **caller** its own isolated port block — a
+Firebase emulator sandbox with fresh data, a Metro port, a private simulator —
+and reclaims it when the caller's process exits.
+
+If you run several Firebase projects locally (dev, tests, CI, agents) you know the
+pain: emulators fight over ports, a bare `firebase emulators:start` collides with
+another suite, and `pkill` "fixes" it by nuking *everyone's* emulators. harbormaster
+removes the collision entirely — two callers that both want "an impulse emulator" get
+two independent suites (own ports, own data), so they never step on each other.
+
+```
+$ eval "$(harbormaster acquire --cwd ./my-app --only auth,firestore --env)"
+$ echo $FIRESTORE_EMULATOR_HOST
+127.0.0.1:11001
+# ...run your tests against the suite...
+# it's torn down automatically when this shell exits
+```
+
+## Requirements
+
+- **Python 3.8+** — the daemon is a single Python script (`#!/usr/bin/env python3`).
+  npm just ships and links the executable; it must be able to find `python3` on PATH.
+- **firebase-tools** (`firebase`) on PATH — it boots the actual emulators.
+- macOS or Linux.
+
+## Install
+
+```bash
+npm install -g harbormaster        # global CLI
+# or add it to a project so it ships with your app/agent:
+npm install harbormaster           # → node_modules/.bin/harbormaster
+```
+
+## Usage
+
+The daemon auto-starts on the first command. `firebase.json` ports are **optional** —
+the daemon injects (and overrides) a pool block into a temp config per lease, so you
+never hand-assign a port.
+
+```bash
+# Boot a private suite for the project in DIR, owned by the calling shell,
+# reaped when it exits. --env prints exports to eval into your shell:
+eval "$(harbormaster acquire --cwd ./my-app --only auth,firestore,functions --env)"
+#   → FIRESTORE_EMULATOR_HOST, FIREBASE_AUTH_EMULATOR_HOST, EMULATOR_<SVC>_PORT,
+#     GCLOUD_PROJECT (from .firebaserc)
+
+# Run a command inside a fresh suite that tears down after (great for CI):
+harbormaster exec --cwd ./my-app --only auth,firestore "npm test"
+
+# Reserve a block WITHOUT booting — for a caller that starts its own suite:
+harbormaster allocate --json          # → { "leaseId": "...", "base": 11000 }
+
+# Have the DAEMON run a long-lived process (Metro, a dev server) as the lease:
+# daemon-owned, so it survives the calling shell / agent session. Liveness is the
+# spawned pid (+ TTL backstop); stop it with `release`. CMD sees METRO_PORT,
+# DETOX_UDID/DETOX_SIM_NAME (with --sim), FBPORTS_BLOCK_BASE, FBPORTS_LEASE_ID.
+eval "$(harbormaster allocate --sim --ttl 604800 \
+        --spawn 'exec npx expo start --port "$METRO_PORT"' --cwd ./my-app --env)"
+harbormaster release "$FBPORTS_LEASE_ID"   # tears down Metro + the sim
+
+# Reuse gate for long-lived sessions: exit 0 iff the lease is FULLY alive
+# (known to the daemon, process running, sim still present, ports up) — anything
+# less means clean up your session state and re-lease:
+harbormaster verify "$FBPORTS_LEASE_ID" || re_lease
+
+harbormaster status                    # everything the daemon has out
+harbormaster down --cwd ./my-app       # release this shell's lease early
+```
+
+`--json` on `acquire`/`allocate` returns the ports/base as JSON for scripting.
+
+Version skew is self-healing: the daemon reports its version on `/health`, and a
+newer client replaces an older daemon in place (leases persist in the ledger;
+every leased process runs in its own session, so nothing is torn down by the
+handover). A stale `node_modules` copy can no longer pin the machine to an old
+daemon.
+
+Leased simulators are part of a lease's liveness: if a sim is deleted externally
+(Xcode cleanup, `simctl delete`, a device reset), the reaper notices within
+`FBPORTS_SIM_CHECK_INTERVAL` (60s) and reaps the lease *whole* — killing its
+spawned process too — so a half-alive session (sim gone, Metro still squatting
+its port) can never linger for callers to "reuse".
+
+## How it works
+
+- **Caller-owned.** One primitive: `acquire {cwd, owner_pid} → {ports, projectId}`.
+  Issuance is unique per *caller*, not per project — so concurrent callers of the
+  same project get independent suites.
+- **Reaped on exit.** The daemon kills the suite (the whole process group, so the JVM
+  children die too) when `owner_pid` dies. The caller's death *is* the release —
+  nothing to remember, and never a reason to `pkill`. A TTL and emulator-liveness are
+  backstops.
+- **Pool ports.** Blocks of 20 from 11000–14999; each firestore suite also gets a
+  private websocket port, so concurrent suites never collide on the default 9150.
+- **No registry, no shared instances.** A "project" is just the `firebase.json` in
+  `cwd`; the project id comes from `.firebaserc`.
+
+See [DESIGN.md](./DESIGN.md) for the full architecture.
+
+## License
+
+MIT — see [LICENSE](./LICENSE).