@mesh-tech/mesh-cli 0.34.3 → 0.35.0

package/skills/{core → mesh}/SKILL.md

@@ -1,29 +1,111 @@
 ---
-name: core
+name: mesh
 description: >
-  The `mesh` developer CLI (@mesh-tech/mesh-cli) — Zitadel/OIDC login,
-  CodeArtifact registry auth, Pulumi deploy/stack passthrough, app scaffolding
-  (create-app), local dev orchestration (dev), SSM tunnels (tunnel), DB access
-  (db), Secrets Manager (secrets), Headscale VPN (vpn), Temporal inspection
-  (temporal), workflow IR extraction (workflow), agent artifact export
-  (artifacts), reading deployed-agent conversations (conversations), and both
-  app API docs and the generated developer portal / CLI reference (docs).
-  Activate when running any `mesh ...`
-  command, scaffolding a tenant app, opening a bastion/DB tunnel, deploying a
-  Pulumi stack via the deployer role, wiring local dev, or building the docs
-  portal.
+  Mesh, the platform, and the `mesh` developer CLI (@mesh-tech/mesh-cli).
+  Explains how an org's Mesh is shaped — the core layer (VPC/EKS/RDS/IAM),
+  the platform layer (Zitadel, Temporal, the Hub, tenant environments) and a
+  personal stack (an isolated platform for one developer on the org's core) —
+  and drives the CLI that sets it up, runs and deploys apps on it, and checks
+  it is current. Activate on ANY Mesh work, even when no command is named:
+  "set me up", "where do I deploy", "deploy my app / studio", a personal
+  stack or platform, mesh login / deploy errors (web identity token could not
+  be validated), local dev (mesh start / mesh dev), scaffolding an app,
+  upgrading Mesh or checking whether an instance is behind, tunnels, DB
+  access, secrets, VPN, Temporal inspection, and the docs portal.
 metadata:
   version: '0.1'
   category: 'tooling'
   type: core
 ---
 
-# `@mesh-tech/mesh-cli`
+# Mesh and the `mesh` CLI
 
 Developer CLI for Mesh Platform. Binary name is **`mesh`**. Install it explicitly —
 `@mesh-tech/app-kit` ships a `mesh` bin shim onto it but declares it an **optional
 peer**, so it is never installed transitively.
 
+## Start here: how an org's Mesh is shaped
+
+Read this before choosing where anything runs. A developer who is asked "where
+do you want to deploy?" without it has no way to answer.
+
+| Layer | What it is | Pulumi program | Registers (SSM) |
+|---|---|---|---|
+| **Core** | The base cloud setup: VPC, EKS cluster, RDS, IAM, DNS zone. One per AWS account/environment the org chooses. | `core/` of the org's platform repo | `/mesh-platform/<tenant>/<env>/core` |
+| **Platform** | Instantiates the services on a core: Zitadel (sign-in), Temporal, the Hub, and the **tenant environments** apps deploy into. Several platforms can share one core. | `platform/` of the same repo | `/mesh-platform/<tenant>/<env>/platform` |
+| **Personal stack** | A platform instance for ONE developer — its own Zitadel, Hub, Temporal and tenant envs, isolated from everyone else's, running on the org's existing core. Nothing you do there can break a shared environment. | a stack of `platform/` (`mesh stack init --platform`) | its own `…/platform` |
+| **App** | Your services, deployed into a tenant environment of a platform. | the app's own Pulumi program | `…/apps/<app>/…` |
+
+`mesh start` is the fifth option: the whole platform locally in Docker, no cloud at all
+(§1b). Use it first when the developer only needs to run an app.
+
+### Two ways orgs lay out environments
+
+Every org decides this for itself, so find out which one yours uses — never assume.
+
+| Shape | The platform repo holds | A personal platform goes |
+|---|---|---|
+| **One core, several platforms** — environments (and developers) are platform stacks sharing a core | `core/Pulumi.acme-dev.yaml` (one core); `platform/Pulumi.acme-dev.yaml`, `platform/Pulumi.acme-qa.yaml`, … — each with `mesh:coreEnv` (and `mesh:coreTenant` when it differs) naming that core | onto that core: another `platform/` stack |
+| **A full core + platform per environment** — each env has its own VPC/EKS, often its own AWS account | `core/Pulumi.acme-dev.yaml` + `core/Pulumi.acme-prod.yaml`, and the matching `platform/` stacks | onto the **non-production** core (`acme-dev`) — **never prod** |
+
+### "Set me up to deploy" — the flow
+
+1. **Ask which repo holds the org's Mesh platform** (the one with `core/` and
+   `platform/` Pulumi programs; often `<org>-mesh-platform`). Don't guess from
+   memory — the answer differs per org.
+2. **Read its stack files** — `core/Pulumi.*.yaml` and `platform/Pulumi.*.yaml` —
+   and classify the shape above. List the candidate cores: each `core/` stack, with
+   the platforms that already name it through `mesh:coreEnv`.
+3. **Recommend the non-production core and confirm it with the developer** before
+   creating anything. Say what will be created and on which AWS account.
+4. **Create and deploy the personal platform** from that repo's `platform/`:
+   ```bash
+   cd <org-platform-repo>/platform
+   mesh stack init --platform --from acme-dev --parent-zone dev.acme.io
+   #   → stack acme-<github-user>-dev: mesh:coreEnv = the base's env, mesh:clusterOwner false,
+   #     org-wide singletons (package registry, VPN) off; hosts become <sub>.<user>-dev.dev.acme.io
+   mesh deploy preview --stack acme-<github-user>-dev
+   mesh deploy up --stack acme-<github-user>-dev
+   ```
+   A platform stack carries no `mesh:deployerRole`: `mesh deploy` runs it with the
+   developer's own AWS credentials **in the core's account** (`AWS_PROFILE`, or the
+   stack's `aws:profile`). A developer with no access to that account needs the
+   platform admin to grant it first — say so rather than retrying.
+5. **Register a tenant environment and point the app at it.** `mesh tenant add <tenant>
+   --stack acme-<github-user>-dev`, deploy again, then in the app's stack file:
+   `mesh:platform` `{name: acme, env: <github-user>-dev}` and `mesh:deployerRole`
+   `arn:aws:iam::<core account>:role/<tenant>-<github-user>-dev-apps-deployer`.
+   Sign in to the personal platform's own Zitadel — `mesh login <user>-dev.dev.acme.io`
+   (its domain) — and put that same context in the app stack as
+   `mesh:loginContext`, so `mesh deploy` uses that session.
+
+### Keeping an instance current (run at session start)
+
+The developer owns keeping their instance up to date; your part is to notice.
+When the session starts in a platform repo or an app directory with a stack, run:
+
+```bash
+mesh stack status        # pinned vs deployed vs latest, per layer; plus this CLI vs latest
+```
+
+It is read-only. When a layer is `behind`, or a pin was never deployed, **tell the
+developer and offer the upgrade — do it only when they say yes.** An upgrade bumps
+`@mesh-tech/infra-components` (and `@mesh-tech/hub` in `platform/`) in **both**
+`core/` and `platform/`, then deploys **both** — core first, previewed, since it is
+the VPC/EKS/RDS. Deploying `platform/` alone leaves core reporting the old version.
+`mesh stack status` prints the exact steps. A deployed version only changes on a
+**successful** deploy, so a failed `up` still reads as the old version.
+
+### Signing in: one session per platform
+
+Each platform has its own Zitadel, and a deployer role trusts only its own
+platform's tokens. `mesh deploy`, `mesh dev` and `mesh kubectl` pick the session
+from the **stack**: `mesh:loginContext` when set, else the stack's `mesh:platform`
+(`{name}.{env}`). The error `AssumeRoleWithWebIdentity … web identity token could
+not be validated` means you are signed in to a *different* platform than the stack
+deploys to — run the `mesh login <context>` the preflight names, never AWS keys.
+`mesh registry login` (packages) is separate and needs no platform at all.
+
 ## Running it
 
 **In this monorepo the CLI runs from TypeScript source — there is no build step and
@@ -99,9 +181,9 @@ Top-level commands (each `register*` is wired in `src/program.ts`, which `bin/me
 | `mesh create-app` | — | Scaffold a tenant app. Composable mode (primitives) or legacy template mode. Interactive when TTY + missing flags. Run at the root of an **empty** `{tenant}-mesh-apps` git clone (a `.git` with no `package.json`/`apps/`/`tenants/` and no enclosing pnpm workspace), it first generates the repo-level shape — `pnpm-workspace.yaml` over `apps/*`, private root `package.json`, scoped-registry `.npmrc`, base `tsconfig.json`, `.gitignore`, `README.md`, `apps/` — then scaffolds the app. Never overwrites an existing file. | `--tenant`, `--name`, `--primitives <csv>` (`service,database,temporal,bucket`), `--template <workflow\|api-auth\|api-role-gating\|external-service>`, `--test` (writes to `tests/tenants/`) |
 | `mesh app` | `check [apps...]` | Checks tenant apps against the **Mesh app contract** (`@mesh-tech/app-kit#apps` → `references/app-contract.md`): a UI is its own Service (`UI_IN_API`, `NO_UI_SERVICE`), the app registers with the Hub (`NO_REGISTER`), sign-in goes through the platform proxy (`CUSTOM_SESSION_AUTH`), logs go through `@mesh-tech/logger` (`NON_PLATFORM_LOGGER`), and people/roles/keys are surfaced through the Hub (`NO_AUTHZ_POINTER`, `METADATA_UNPUBLISHED`, `IAC_GRANTS`, `PASSWORD_STORE`, `EMAIL_ALLOWLIST`, `USERS_TABLE`), and a docs site is published through the Hub docs door rather than on a hostname of its own, with its readers stated and grantable (`DOCS_SITE_PUBLIC` — advisory: the public hostname is the deprecated compatibility path and keeps deploying, so the finding names the door instead — `DOCS_AUDIENCE_IMPLICIT`, `DOCS_AUDIENCE_UNGRANTABLE`, `DOCS_VENDOR_AUDIENCE_PARTNER` — a vendor surface's reference is never partner-readable); a tenant-local auth lib is advisory (`LOCAL_AUTH_LIB`), as is a pointer that names its own Zitadel project instead of binding the env with `zitadelAppProjectId` (`POINTER_PROJECT_OVERRIDE`). Gate 0.6 additionally resolves the Pulumi program's imports against the installed tree (`UNRESOLVABLE_IMPORT`) — an optional peer nothing declares installs silently and `skipLibCheck` hides it, so the break otherwise waits for `mesh deploy`; with no `node_modules` the gate reports itself not checked rather than met. One check per code on the `mesh dev doctor` engine; exit 1 on any BLOCK. The SAME gate `mesh create-app` runs after scaffolding, tenant CI runs beside `mesh skills sync --check`, and the platform reviewer runs on a PR. Default: every `apps/*` with a Pulumi program. | `[apps...]` (repo-relative dirs), `--root <dir>`, `--json`, `--verbose` |
 | `mesh init` | (bare = wizard), `app-tenant`, `platform <tenant>` | **Bare `mesh init` is the guided setup wizard**: tenant (pre-filled from `mesh.json` / Pulumi `mesh:tenant`), **where the tenant runs — local-only, a deployed platform, or NOTHING YET (scaffold one; §1c)**, registry access (in-process `mesh registry login`; a 403 marks the step ✘ and the wizard continues), platform sign-in on the deployed branch only, repo bootstrap (empty clone, or an empty non-git folder the wizard offers to `git init`) or shape check, skills sync; writes `mesh.json` (`{tenant, platform}`) so `create-app` / `mesh dev` default `--tenant`; flags `--tenant --local\|--platform <env>\|--new-platform [env] --device --profile --skip-repo --yes --json`, plus the scaffold's own under `--new-platform` (`--region --domain --state-bucket --create-state-bucket/--no-create-state-bucket --dir`); exit 1 if any step failed, 130 on Ctrl-C. `platform <tenant>` **prompts** on a TTY for env/region/profile/state bucket (defaults on each) and for `--domain`, which has **no** default because a placeholder scaffolds an unvalidatable zone; it offers to create the Pulumi state bucket (versioned) and prints the deploy sequence including the manual NS delegation. `app-tenant` is the repo doctor (A1–A3): read-only check pipeline — CLI auth, registry access, public npm not hijacked, platform reachable (Hub API), app-tenant registration, deployer role, repo shape, agent skills — each failure names the exact `mesh` fix. **Registry access** is a live probe of the CodeArtifact token, not a grep of `~/.npmrc`: an expired token `fail`s (it used to `pass`), an unreachable registry `warn`s (a network fault is not an auth fault). **Public npm not hijacked** fails when `~/.npmrc` carries an unscoped `registry=<codeartifact>` line, which makes CodeArtifact your default registry — one `mesh registry login` fixes both. Exit 0 iff all pass; re-run any time. `--fix` applies developer-scope fixes (registry token, skills sync); operator actions stay remediations — and `--profile <p>` is the SSO profile `--fix` logs into CodeArtifact with, echoed back in the `fix:` command each check prints so the advertised command is the one that runs. | `app-tenant`: `--tenant <name>`, `--context <ctx>` (default `local`), `--hub-url <url>`, `--fix`, `--profile <sso>`, `--json` |
-| `mesh skills` | `sync` | Agent-skill distribution (D1/D2): installs the base building-with-Mesh skills into the repo's `.claude/skills/mesh-*/` (managed-marker files) and wires TanStack-Intent discovery for the pattern skills shipping inside `@mesh-tech/*` packages (`.intent/hooks/` gate + `.claude/settings.json` SessionStart hook + AGENTS.md fence — the PR #2356 mechanism, vendored). Idempotent; runs automatically after `mesh create-app`. | `sync`: `--check` (CI/doctor: exit 1 when missing/stale), `--root <path>` |
+| `mesh skills` | `sync` | Agent-skill distribution (D1/D2): installs the base building-with-Mesh skills into the repo's `.claude/skills/` — this one as `mesh/` (`/mesh`) — and the `@mesh-tech/*` package skills as `mesh-<pkg>[-<domain>]/` (`-core` dropped: `mesh-authn`, `mesh-app-kit-apps`); every copy's `name:` is its directory, and managed copies left under older names (`mesh-core/`, `mesh-<pkg>-core/`) are removed and wires TanStack-Intent discovery for the pattern skills shipping inside `@mesh-tech/*` packages (`.intent/hooks/` gate + `.claude/settings.json` SessionStart hook + AGENTS.md fence — the PR #2356 mechanism, vendored). Idempotent; runs automatically after `mesh create-app`. | `sync`: `--check` (CI/doctor: exit 1 when missing/stale), `--root <path>` |
 | `mesh docs` | `build`, `portal`, `cli-reference` | `build` builds/validates an **app's** OpenAPI specs from `docs/docs.config.json` (file copy or SymXchange generation). `start`/`stop`/`list` are the simple front door: `start` serves the docs on loopback — DETACHED in a tmux session named `mesh-docs` by default (prints the bare URL once live; `stop` kills the session; `--foreground` or any non-TTY runs in the foreground) — serving the working tree in a mesh-platform checkout, or the published `@mesh-tech/docs` artifact fetched from the role-gated CodeArtifact registry anywhere else (docs version == the @mesh-tech/* baseline it describes; `-v` pins, default latest; cached under `~/.cache/mesh/docs/`). `list` shows the published docs versions. `portal` assembles the **Mesh developer portal** (docs.meshtech.io) from every `docs.json`-opted-in doc root in the repo — a directory publishes its markdown iff it holds a `docs.json`; route/title/order/nav derive from the tree and each file, so adding a doc is adding one file — then hands the assembled tree to Zudoku (`apps/docs/`): default runs `zudoku build`, `--serve` runs `zudoku dev`, `--assemble-only`/`-o` stop after assembly. Links between published files are rewritten to routes; links to unpublished repo files become blob links. Reserved directory names (`plans/`, `designs/`, `incidents/`, … the frozen list in `src/docs/schema.ts`) are excluded at any depth. `cli-reference` regenerates `docs/portal/generated/cli-reference.md` from THIS command tree, so it can't document a flag the binary lacks. `portal --check` is the CI gate (`pnpm check:docs-portal`): docs.json schema validation, reserved-segment fail-close, broken links, and CLI-reference staleness; `--diff-base <ref>` prints the publish-set diff for PR job summaries. Hidden commands and `--version` are excluded from the reference. | `build`: `-c/--config <path>` (default `docs/docs.config.json`); `portal`: `-o/--out <dir>`, `--assemble-only`, `--serve`, `-p/--port <n>` (default `3000`), `--check`, `--print-manifest`, `--diff-base <ref>`, `--manifest-out <path>`; `cli-reference`: `-o/--out <path>`, `--check`; `start`: `-v/--version <v>` (default: latest), `-p/--port <n>` (default `4400`; `0` is foreground-only), `--dev` (foreground HMR), `--foreground`; `stop`: no flags; `list`: no flags |
-| `mesh stack` | `init`, `rm <name>` | `init` creates a personal dev stack `dev-{github-user}`, copying base-stack config, setting `mesh:deploy: false`, inheriting KMS secrets provider; `--worktree` appends the git-worktree token (`dev-{user}-{token}`) so concurrent worktrees deploy to **distinct** stacks (SSM paths, Temporal namespace, Nexus endpoint all derive from the stack name → one discriminator isolates all three; no-op on the primary checkout). `rm <name>` removes a personal stack (backend state + local config), credentialed. Both assume the deployer role — no raw `pulumi` / manual creds needed. | `init`: `--from <stack>`, `--name <stack>`, `--worktree`; `rm`: `--yes` |
+| `mesh stack` | `init`, `status`, `rm <name>` | `init` in an **app** creates a personal app stack `dev-{github-user}`, copying base-stack config, setting `mesh:deploy: false`, inheriting KMS secrets provider; `--worktree` appends the git-worktree token (`dev-{user}-{token}`) so concurrent worktrees deploy to **distinct** stacks (SSM paths, Temporal namespace, Nexus endpoint all derive from the stack name → one discriminator isolates all three; no-op on the primary checkout). `init --platform` in a platform repo's **`platform/`** creates a **personal platform** `{tenant}-{github-user}-dev` — its own Zitadel, Hub, Temporal and tenant envs on the base stack's core (`mesh:coreEnv`, `mesh:clusterOwner false`, registry/VPN singletons off, deploy on); `--parent-zone` gives it its own DNS zone. `status` is the read-only drift check: for each platform this directory deploys (or an app deploys to), pinned vs deployed (the `core` and `platform` SSM registrations) vs latest `@mesh-tech/infra-components` / `@mesh-tech/hub`, plus this CLI vs the latest `@mesh-tech/mesh-cli`, with the upgrade steps when anything is behind. `rm <name>` removes a personal stack (backend state + local config), credentialed. `init`/`rm` assume the deployer role — no raw `pulumi` / manual creds needed. | `init`: `--from <stack>`, `--name <stack>`, `--worktree`, `--adopt`, `--platform`, `--parent-zone <zone>`; `status`: `--stack`, `--profile`, `--region`, `--json`, `--check` (exit 1 when behind); `rm`: `--yes` |
 | `mesh tenant` | `add <name>`, `list` | Register/inspect app tenants on a platform stack. Run from the **platform layer** of a tenant platform repo (e.g. `mesh-sandbox/platform`). `add` inserts the tenant under `mesh:tenants` in `Pulumi.<stack>.yaml` via a comment-preserving YAML round-trip (untouched lines don't reformat), refuses a tenant that is already declared (an existing entry may carry fields a re-add would drop), derives the `{name}-{env}` subdomain from the stack name (echoed in the success line), and guards against running on the core layer or an app dir; it then prints the deploy + verify sequence — `mesh deploy up` is what makes the registration live. `list` prints what the stack config declares, so "is it registered?" is answerable without opening YAML (warns when the project doesn't look like a platform layer, so an empty result isn't mistaken for "no tenants"). | `add`: `--display-name <name>`, `--subdomain <sub>`, `--zitadel-org-id <id>`, `--stack <stack>`, `--json`; `list`: `--stack <stack>`, `--json` |
 | `mesh tunnel [group]` | `external <name>` | SSM port-forward to platform services. Groups: `dev` (temporal+db), `temporal`. **`external <name>`** tunnels to a *registered* `ExternalService`: host/port are read from its credential secret (`mesh/{app-tenant}/{app-stage}/external/{name}`) — the same values deployed apps resolve at runtime — so anything the bastion can reach is tunnelable with no per-service CLI entry and no bastion services-map change. `-t/-e` are **optional inside an app directory** — they fall back to the same Pulumi/SST context detection `mesh dev` uses, which fails closed elsewhere, so pass `-t` outside an app dir (`-t` alone is enough; `-e` alone is not). `-e` names a platform env, not a stack, so it no longer doubles as the stage hint: the credential axis `external` keys on stays the app's real stack name. And `AWS_REGION`/`AWS_PROFILE` default from the nearest Pulumi config + the `{tenant}-{env}` profile convention (region falls back to the effective profile's own `region` — the `[default]` profile when none was resolved — before the platform default us-east-2). | root: `-t/--tenant`, `-e/--env`, `-l/--list`, `-s/--services <csv\|group>`, `--db-port`, `--temporal-ui-port`, `--temporal-frontend-port`; `external`: `-p/--port` (default: the remote port), `-k/--key <key>` (multi-instance `credentials.keyedBy` externals), `--app-tenant`/`--app-stage` (credential axis; default to `-t`/`-e`) |
 | `mesh secrets` | `exec <command...>`, `set [service]`, `reindex [service]`, `migrate-config` | Manage external-service creds in Secrets Manager. `exec` injects secrets as env; `set` reads field schema from SSM and prompts (or `--json`) — with `--key` it also records the instance in the service's instance index (`{prefix}/.index`, what `listInstances()` reads). `reindex` rebuilds that index from the per-key secrets actually in Secrets Manager (operator-credential `ListSecrets`) — the repair for an index that drifted (out-of-band secret create/delete, concurrent writers, malformed value) and the populate step when rolling the index out to an estate: **deploy the declaring stack first** (creates the managed, empty index), then `reindex`. All three resolve tenant + platform env from the cwd stack file and the app's `config.ts`; `-t/--tenant` overrides the tenant for one invocation, and with several `Pulumi.<stack>.yaml` files present pass `--stack` rather than relying on the base-stack pick. | `set`: `--key <id>` (multi-instance), `--all`, `--json <str>`, `-t/--tenant`, `--stack`, `--region`; `reindex`: `--dry-run`, `-t/--tenant`, `--stack`, `--region`; `migrate-config`: `--force`, `--dry-run`, `-t/--tenant`, `--stack`, `--region` |
@@ -265,59 +347,12 @@ it over the containerized Hub when you are iterating on Hub code and want the UI
 reload on save. Both default to :9000, so run them together only with `--port` /
 `MESH_HUB_DEV_PORT` set on this one.
 
-### 3. Concurrent `mesh dev` across git worktrees and sibling apps (auto-isolation)
+### 3. Concurrent `mesh dev` across git worktrees and sibling apps
 
-Run the **same plain `mesh dev`** from each worktree's app dir — there is nothing special to
-configure. `mesh dev` derives a deterministic token from the worktree path and threads it through
-the tmux session name, the tmpdir session-state + env-file dir and the Temporal task-queue suffix, so
-two worktrees don't kill each other's session — as long as their directory basenames differ, since the
-session name carries the worktree's sanitized basename, not the hash. The reserved **service-port
-block is derived from (worktree, app)**, not the worktree alone — a repo can hold several apps whose
-sessions run concurrently (e.g. an apps repo's `worker-a` + `worker-b`), and one
-block per worktree handed both the same ports.
-
-```bash
-# worktree A
-cd .worktrees/feature-a/tests/tenants/acme/apps/demo-agent   # monorepo test tenant
-mesh dev --dry-run          # preview: session name, port block, state/env paths, task queue
-mesh dev                    # launch (attaches tmux; --headless to skip)
-
-# worktree B — SAME command, different shell; auto-isolated
-cd .worktrees/feature-b/tests/tenants/acme/apps/demo-agent
-mesh dev
-
-# sibling app in worktree A — same worktree, still its own port block
-cd .worktrees/feature-a/tests/tenants/acme/apps/silverlake-demo
-mesh dev
-```
-
-- The **primary checkout** keeps its historical identity (`${project}-dev` session, the app's
-  preferred ports). Only **linked worktrees** get an offset session name + a port block in the 40000+
-  range. `mesh dev --dry-run` labels which you are (`primary checkout` vs `linked worktree (block N)`).
-- **Port blocks are disjoint by default, not by construction.** The block is a hash bucketed into 63
-  slots, so two `(worktree, app)` pairs *can* land on the same block; the in-block free-port probe and
-  the ephemeral-port fallback are the backstop when they do — but that backstop is the same
-  check-then-bind probe that raced two still-booting sessions onto one port before #2889, so treat a
-  shared block as a live hazard, not a handled case. Two consequences worth knowing before you
-  debug a collision as a regression:
-  - Sibling apps in the **primary checkout** are deliberately excluded — they keep preferred-port
-    semantics (block 0) and rely on that probe, so run them from linked worktrees if you need them
-    isolated. This is the layout most tenant app repos use day to day.
-  - App-scoping raises the number of hashed entities from *#worktrees* to *#worktrees × #apps*, so
-    bucket collisions get likelier as you add apps.
-
-  A deterministic, collision-free allocator that also folds primary checkouts back in is planned.
-- **Do NOT force a shared `--session`** across worktrees — that defeats the auto-isolation and
-  reintroduces the collision. Omit `--session` and let it derive per worktree.
-- `mesh dev --kill` / `--status` act only on the current worktree's session.
-- If the stack was last deployed from a *different* worktree, `mesh dev` **refuses** to launch a service
-  whose source resolves outside the current worktree (guards against silently running stale code) — re-run
-  `mesh deploy up` in this worktree.
-- **Deploying isolated stacks per worktree?** The `mesh dev` isolation above is local-only. To
-  `mesh deploy up` a `deploy:false` personal stack from several worktrees at once, give each worktree
-  its own stack with `mesh stack init --worktree` — otherwise they collide on SSM export paths
-  (`ParameterAlreadyExists`) and the Temporal namespace (both derive from the stack name).
-  `mesh deploy up`/`destroy` **warns** when it sees a shared (non-worktree) stack name in a linked worktree.
+Run the **same plain `mesh dev`** in each worktree — it auto-isolates the tmux session, state
+dir, port block and Temporal task queue. Don't force a shared `--session`. To *deploy* from
+several worktrees at once, give each its own stack: `mesh stack init --worktree`. Details,
+port-block caveats and the stale-source guard: [`references/worktrees.md`](references/worktrees.md).
 
 ### 4. Deploy (pulumi passthrough)
 
@@ -431,6 +466,7 @@ re-reads stack/SSM output; if the *topology* changed, run `mesh deploy up` first
 | `No mesh:deployerRole found` | stack config missing the role | `mesh deploy config set mesh:deployerRole <arn>` |
 | `Could not determine stack` (deploy) | no selected stack & no `--stack` | `mesh deploy stack select <name>` or pass `--stack` |
 | `artifacts get` → ECONNREFUSED / 404 | agent API down or convo expired | start API (`mesh dev`) or pass `--api-url`; restart API after code changes |
+| `AssumeRoleWithWebIdentity … web identity token could not be validated` (`mesh deploy`, `mesh dev --stack`) | the cached session is from a different platform's Zitadel than the one the stack's deployer role trusts — e.g. a `mesh.dev` session (the package-registry front door) against a stack on the `mesh`/`dev` platform, which signs in as `mesh-platform.dev` | run the `mesh login <context> --device` the preflight prints (it is derived from the stack: `mesh:loginContext`, else `mesh:platform`); for a personal platform, set `mesh:loginContext` to its domain. Never fall back to exported AWS keys |
 | `login` config not found | context not discoverable | a known first-party short context (`mesh.dev`) resolves anonymously — no AWS needed. Otherwise use the platform's full domain (`mesh login dev.<tenant>.meshtech.io`). Only if neither works: ensure AWS SSO is active + the platform is deployed (SSM `cliClientId`), or add config to `~/.config/mesh/config.json` |
 | `vpn`/`temporal --show-payloads` fail | kubeconfig not set / expired creds | refresh the tenant cluster kubeconfig (`mesh vpn connect <ctx>` wires it; kubeconfig absorption into `mesh deploy` is planned) then retry | <!-- skill-lint-disable-next-line — next row's backticked `kubectl …` is a quoted ERROR SYMPTOM, not an operator command; the row's fix column already routes through mesh dev -->|
 | `kubectl could not read a kubeconfig … macOS blocks it` (from `mesh kubectl`/`logs`/`exec`/`temporal` in a bare shell) | macOS Full-Disk-Access blocks reading `~/.kube` from a shell outside the `mesh dev` session | run the command **inside** the `mesh dev` tmux window, or `export KUBECONFIG=$TMPDIR/mesh-dev-sessions/<session>.kubeconfig` first |
@@ -447,39 +483,9 @@ re-reads stack/SSM output; if the *topology* changed, run `mesh deploy up` first
 
 ## Programmatic API
 
-`@mesh-tech/mesh-cli` re-exports its utils (`src/utils/index.ts`):
-
-```ts
-import { detectContext, getPlatformBastionInfo } from "@mesh-tech/mesh-cli";
-const ctx = detectContext();                                  // { tenant, stage, ... }
-const bastion = await getPlatformBastionInfo(ctx.tenant, ctx.platformEnv);
-```
-
-Exports include `detectContext`, `getPlatformBastionInfo` (bastion), credentials,
-pulumi, vpn, and log helpers.
-
-`detectContext(stageArg?, { tenant?, stack? })` fails closed: when no source
-declares a tenant it throws a `MeshCliError` naming what it looked at, instead
-of silently defaulting to the hub tenant `"mesh"`. The shared `config.ts` must
-export the documented shape — `export const config = { tenant: … }` or
-`export default { … }`; a renamed or re-exported const is not scanned, and a
-`tenant:` in a comment, in a nested ref, or in a second export cannot supply
-one. `mesh tunnel` and the `secrets` commands forward their `--tenant` through
-`options.tenant`; `db` subcommands take `--tenant` and `--env` together,
-bypassing detection. Embedders should pass `options.tenant` or catch the error.
-
-The two options are not interchangeable. `options.stack` is a real stack
-selection — it names the one `Pulumi.<stack>.yaml` allowed to supply
-`mesh:tenant`, and no match means none is read. `stageArg` is only a hint:
-`mesh tunnel -e <env>` passes a platform env, which names no stack file, so
-tunnel passes no hint at all and lets the stack file supply the stage.
-
-The returned `platformEnv` is `envMap[stage] ?? defaultEnv` from the config's
-`platform` block — the same expression the config's own `getPlatform()`
-evaluates, so the CLI addresses the environment the stack was deployed
-against. `detectedTenant` carries what detection produced before any override,
-and a `--tenant`/`MESH_TENANT` that disagrees with it warns: paths follow the
-override, implicit AWS credentials do not.
+`@mesh-tech/mesh-cli` re-exports its utils (`detectContext`, `getPlatformBastionInfo`, credentials,
+pulumi, vpn, log helpers). `detectContext` fails closed when no source declares a tenant. Full
+contract: [`references/programmatic-api.md`](references/programmatic-api.md).
 
 ## See also
 

package/skills/mesh/references/programmatic-api.md

@@ -0,0 +1,35 @@
+# Programmatic API
+
+`@mesh-tech/mesh-cli` re-exports its utils (`src/utils/index.ts`):
+
+```ts
+import { detectContext, getPlatformBastionInfo } from "@mesh-tech/mesh-cli";
+const ctx = detectContext();                                  // { tenant, stage, ... }
+const bastion = await getPlatformBastionInfo(ctx.tenant, ctx.platformEnv);
+```
+
+Exports include `detectContext`, `getPlatformBastionInfo` (bastion), credentials,
+pulumi, vpn, and log helpers.
+
+`detectContext(stageArg?, { tenant?, stack? })` fails closed: when no source
+declares a tenant it throws a `MeshCliError` naming what it looked at, instead
+of silently defaulting to the hub tenant `"mesh"`. The shared `config.ts` must
+export the documented shape — `export const config = { tenant: … }` or
+`export default { … }`; a renamed or re-exported const is not scanned, and a
+`tenant:` in a comment, in a nested ref, or in a second export cannot supply
+one. `mesh tunnel` and the `secrets` commands forward their `--tenant` through
+`options.tenant`; `db` subcommands take `--tenant` and `--env` together,
+bypassing detection. Embedders should pass `options.tenant` or catch the error.
+
+The two options are not interchangeable. `options.stack` is a real stack
+selection — it names the one `Pulumi.<stack>.yaml` allowed to supply
+`mesh:tenant`, and no match means none is read. `stageArg` is only a hint:
+`mesh tunnel -e <env>` passes a platform env, which names no stack file, so
+tunnel passes no hint at all and lets the stack file supply the stage.
+
+The returned `platformEnv` is `envMap[stage] ?? defaultEnv` from the config's
+`platform` block — the same expression the config's own `getPlatform()`
+evaluates, so the CLI addresses the environment the stack was deployed
+against. `detectedTenant` carries what detection produced before any override,
+and a `--tenant`/`MESH_TENANT` that disagrees with it warns: paths follow the
+override, implicit AWS credentials do not.

package/skills/mesh/references/worktrees.md

@@ -0,0 +1,53 @@
+# Concurrent `mesh dev` across git worktrees and sibling apps
+
+Run the **same plain `mesh dev`** from each worktree's app dir — there is nothing special to
+configure. `mesh dev` derives a deterministic token from the worktree path and threads it through
+the tmux session name, the tmpdir session-state + env-file dir and the Temporal task-queue suffix, so
+two worktrees don't kill each other's session — as long as their directory basenames differ, since the
+session name carries the worktree's sanitized basename, not the hash. The reserved **service-port
+block is derived from (worktree, app)**, not the worktree alone — a repo can hold several apps whose
+sessions run concurrently (e.g. an apps repo's `worker-a` + `worker-b`), and one
+block per worktree handed both the same ports.
+
+```bash
+# worktree A
+cd .worktrees/feature-a/tests/tenants/acme/apps/demo-agent   # monorepo test tenant
+mesh dev --dry-run          # preview: session name, port block, state/env paths, task queue
+mesh dev                    # launch (attaches tmux; --headless to skip)
+
+# worktree B — SAME command, different shell; auto-isolated
+cd .worktrees/feature-b/tests/tenants/acme/apps/demo-agent
+mesh dev
+
+# sibling app in worktree A — same worktree, still its own port block
+cd .worktrees/feature-a/tests/tenants/acme/apps/silverlake-demo
+mesh dev
+```
+
+- The **primary checkout** keeps its historical identity (`${project}-dev` session, the app's
+  preferred ports). Only **linked worktrees** get an offset session name + a port block in the 40000+
+  range. `mesh dev --dry-run` labels which you are (`primary checkout` vs `linked worktree (block N)`).
+- **Port blocks are disjoint by default, not by construction.** The block is a hash bucketed into 63
+  slots, so two `(worktree, app)` pairs *can* land on the same block; the in-block free-port probe and
+  the ephemeral-port fallback are the backstop when they do — but that backstop is the same
+  check-then-bind probe that raced two still-booting sessions onto one port before #2889, so treat a
+  shared block as a live hazard, not a handled case. Two consequences worth knowing before you
+  debug a collision as a regression:
+  - Sibling apps in the **primary checkout** are deliberately excluded — they keep preferred-port
+    semantics (block 0) and rely on that probe, so run them from linked worktrees if you need them
+    isolated. This is the layout most tenant app repos use day to day.
+  - App-scoping raises the number of hashed entities from *#worktrees* to *#worktrees × #apps*, so
+    bucket collisions get likelier as you add apps.
+
+  A deterministic, collision-free allocator that also folds primary checkouts back in is planned.
+- **Do NOT force a shared `--session`** across worktrees — that defeats the auto-isolation and
+  reintroduces the collision. Omit `--session` and let it derive per worktree.
+- `mesh dev --kill` / `--status` act only on the current worktree's session.
+- If the stack was last deployed from a *different* worktree, `mesh dev` **refuses** to launch a service
+  whose source resolves outside the current worktree (guards against silently running stale code) — re-run
+  `mesh deploy up` in this worktree.
+- **Deploying isolated stacks per worktree?** The `mesh dev` isolation above is local-only. To
+  `mesh deploy up` a `deploy:false` personal stack from several worktrees at once, give each worktree
+  its own stack with `mesh stack init --worktree` — otherwise they collide on SSM export paths
+  (`ParameterAlreadyExists`) and the Temporal namespace (both derive from the stack name).
+  `mesh deploy up`/`destroy` **warns** when it sees a shared (non-worktree) stack name in a linked worktree.

package/skills/pull-remote-agent-context/SKILL.md

@@ -187,7 +187,7 @@ fub | jq ...`) rather than pass `--json` — output auto-detects non-TTY.
 
 ## See also
 
-- `packages/mesh-cli/skills/core/SKILL.md` — full `mesh` command reference,
+- `packages/mesh-cli/skills/mesh/SKILL.md` — full `mesh` command reference,
   including the `conversations`/`artifacts`/`temporal` rows this skill
   builds on.
 - `packages/agent-remote/agent-targets.example.json` — a worked registry

package/templates/api-auth/README.md.hbs

@@ -58,8 +58,9 @@ that Service's own ingress hostname; a Service that answers for more than one
 hostname (`ingress.extraHostnames`) sets `auth.deriveRedirectUrl: true` so each
 host gets its own callback, and registers a redirect URI per host at the IdP.
 The proxy keeps each session in its cookie, which grows with the user's roles;
-once a user holds enough of them the load balancer rejects every request with a
-`400`. `auth.sessionStore: "redis"` keeps sessions in a small Redis the Service
+once a user holds enough of them every request fails — a `431` from the app,
+which receives the cookie too, or a `400` from the load balancer.
+`auth.sessionStore: "redis"` keeps sessions in a small Redis the Service
 creates for itself, so the cookie stays under 1KB.
 
 `index.ts` owns the whole access surface, so there is nothing to create by