@dooer/dooer-test-env 1.16.0 → 1.17.0

package/docs/getting-access.md

@@ -1,27 +1,33 @@
 # Getting access (new developer)
 
-`dooer-test-env` reads the staging cluster to build its compose file, log in to the image registries and
-fetch the BankID cert. That needs two things, in this order:
+`dooer-test-env` builds your local stack from the staging environment: the service list, the registry
+credentials and the BankID material all come from there. That needs two things, in this order:
 
-1. **VPN** — the cluster API is not reachable from the open internet.
-2. **kubectl access** — a client certificate identifying you to the cluster.
+1. **VPN** — nothing this tool talks to is reachable from the open internet.
+2. **An admin session for staging** — `remote-environment login`.
 
-**Both come from an admin**: they create your VPN tunnel and your cluster credential, and send you two
-files — a WireGuard `.conf` and a `kubeconfig`. You install them and check they work. See
-[issuing-access.md](./issuing-access.md) for their side.
+**kubectl is no longer required.** It used to be: every developer held a cluster certificate that could
+read every Secret in the namespace. That is now behind
+[`service-dooer-test-env`](../_specification/service-dooer-test-env.md), which exposes only the operations
+this tool needs, each admin-gated and recorded. If you operate the cluster itself you may still want a
+kubectl credential — see [issuing-access.md](./issuing-access.md) — but you do not need one to use this
+tool.
+
+**The VPN config comes from an admin**: they create your tunnel and send you a WireGuard `.conf`.
 
 ---
 
 ## 0. Prerequisites
 
 ```bash
-brew install wireguard-tools kubernetes-cli
+brew install wireguard-tools
 ```
 
-`openssl` ships with macOS. Check everything is there:
+Plus Docker (Desktop) and Node 18+. `kubernetes-cli` is no longer needed for this tool — install it only
+if you operate the cluster for other reasons.
 
 ```bash
-wg --version && kubectl version --client && openssl version
+wg --version && docker info >/dev/null && node --version
 ```
 
 ---
@@ -90,66 +96,23 @@ amount of kubectl configuration will help.
 
 ---
 
-## 2. kubectl access
-
-**Your admin sends you a ready-made `kubeconfig` file.** You do not generate keys or assemble anything —
-they create the credential and hand it over
-([issuing-access.md § 2](./issuing-access.md#2-kubectl--create-their-credential)).
-
-Like the VPN config, that file contains a **private key**. Same handling: install it, then delete the copy
-they sent you.
-
-### 2a. Install it
-
-**If you have no other clusters** — simplest, just put it in place:
-
-```bash
-mkdir -p ~/.kube
-cp ~/Downloads/<yourname>.kubeconfig ~/.kube/config
-chmod 600 ~/.kube/config
-rm ~/Downloads/<yourname>.kubeconfig
-```
-
-**If you already use kubectl for something else**, merge instead of overwriting:
-
-```bash
-cp ~/.kube/config ~/.kube/config.backup            # always back up first
-KUBECONFIG=~/.kube/config:~/Downloads/<yourname>.kubeconfig \
-  kubectl config view --flatten > /tmp/merged && mv /tmp/merged ~/.kube/config
-chmod 600 ~/.kube/config
-rm ~/Downloads/<yourname>.kubeconfig
-```
+## 2. Log in to staging
 
-Then select the context (the admin will tell you its name, `<firstname>-admin@kubernetes`):
+Everything the tool used to read with kubectl now comes from `service-dooer-test-env`, authorized by your
+own session:
 
 ```bash
-kubectl config get-contexts
-kubectl config use-context <firstname>-admin@kubernetes
+npx @dooer/dooer-test-env@latest remote-environment login --env staging --email you+admin@dooer.com
+npx @dooer/dooer-test-env@latest remote-environment status --env staging
 ```
 
-### 2b. Verify
+It has to be an **admin** account, and therefore **email + password** — BankID cannot reach an admin
+account at all (`loginWithBankIdV2` is pinned to `user_type: 'customer'` server-side, and the partner flow
+resolves your `hi` user). `status` prints which of your accounts you ended up as; read the **user type**
+line.
 
-With the VPN up:
-
-```bash
-kubectl auth whoami
-# Username  <firstname>-admin
-# Groups    [system:authenticated]
-
-kubectl get pods -n dooer-staging | head -3
-kubectl get pods -n dooer-production | head -3
-```
-
-Then the two checks `dooer-test-env` actually depends on:
-
-```bash
-kubectl auth can-i get secrets -n dooer-staging     # yes — registry pull secrets + BankID cert
-kubectl get svc dooer-database -n dooer-staging     # the database service the base-DB build reads
-```
-
-All of those must succeed before `dooer-test-env setup` will work.
-
----
+Don't have an admin account? Ask someone who does to create one —
+[user-management.md](./user-management.md) covers `user create`.
 
 ## 3. You are done
 
@@ -157,7 +120,8 @@ All of those must succeed before `dooer-test-env setup` will work.
 npx @dooer/dooer-test-env@latest setup
 ```
 
-`setup` re-checks all of the above and tells you exactly which piece is missing if something is wrong.
+`setup` re-checks all of the above and tells you exactly which piece is missing if something is wrong. It
+needs Docker running, the VPN up, and the session from section 2.
 
 ---
 
@@ -165,11 +129,12 @@ npx @dooer/dooer-test-env@latest setup
 
 | Symptom | Cause |
 | --- | --- |
-| `dial tcp … i/o timeout` from kubectl | VPN is down — `sudo wg show`, then restart it (1b) |
+| `cannot reach service-dooer-test-env … is the VPN up?` | VPN is down — `sudo wg show`, then restart it (1b) |
 | `nc` to the API server hangs | Tunnel is up but not carrying traffic; restart it |
-| `error: You must be logged in to the server (Unauthorized)` | The certificate in your kubeconfig is expired or unknown to the cluster — ask for a re-issue |
-| `Error from server (Forbidden)` | You *are* authenticated; your user has no permissions bound yet — ask the admin to complete their § 2e (bind permissions) |
+| `not logged in to staging` | Run the login in section 2 |
+| `your staging session expired …` | Sessions are short-lived; log in again |
+| `needs an "admin" session` | You are signed in as your customer or `hi` account — log in with your admin email |
 | `wg-quick: 'roboten' already exists` | The interface is already up — `sudo wg-quick down roboten` first |
 
-Certificates are issued for **one year**. When yours expires, ask your admin for a new kubeconfig and
-repeat section 2 — your permissions stay in place, so they only have to re-issue the credential.
+Nothing here expires on a yearly cycle any more: your session is short-lived and you simply log in again.
+The VPN tunnel is the only thing an admin has to issue, and it is issued once.

package/docs/issuing-access.md

@@ -7,6 +7,15 @@ The other side of [getting-access.md](./getting-access.md). Two independent thin
 | **VPN tunnel** | you | create it and send them the config |
 | **kubectl credential** | you | create the certificate, bind permissions, send a kubeconfig |
 
+> **Using `dooer-test-env` no longer requires §2.** Developers need the VPN (§1) and an admin session —
+> nothing more. Every cluster read the tool used to do is now behind `service-dooer-test-env`, which
+> exposes a fixed set of admin-gated, audited operations instead of an open-ended kubeconfig
+> ([the specification](../_specification/service-dooer-test-env.md) explains the reasoning).
+>
+> Issue a cluster certificate only to people who **operate the cluster** — deploying, editing manifests,
+> restarting pods. That is a much smaller group, and it is the point: a kubeconfig grants arbitrary
+> operations, so the fewer that exist, the smaller the surface.
+
 You need a Robo10 token (§1a), `kubectl` with cluster-admin (`system:masters`), and your own VPN up.
 
 ---

package/lib/bankid.js

@@ -68,6 +68,15 @@ function keychainDelete(key) {
   return sh('security', ['delete-generic-password', '-a', KEYCHAIN_ACCOUNT, '-s', keychainService(key)]).status === 0
 }
 
+// Pull the BankID material from service-dooer-test-env — no kubectl. Same values, same keys.
+async function pullFromService(envName = 'staging') {
+  const remote = require('./remote')
+  const serviceClient = require('./service-client')
+  const env = remote.resolveEnv(envName)
+  const { values } = await serviceClient.request(env, { path: '/v1/bootstrap/bankid' })
+  return values
+}
+
 // Pull the bankid secrets out of the staging k8s secret via kubectl. k8s stores each value base64-encoded
 // in `.data`; we decode to the raw value the staging container would receive (for the PFX that raw value is
 // itself the base64 PFX string, exactly what @dooer/config's `Buffer.from(pfx,'base64')` expects).
@@ -87,7 +96,24 @@ function pullFromStaging({ namespace = DEFAULT_NAMESPACE, secret = DEFAULT_SECRE
 }
 
 // Pull + store in the keychain. Returns the list of keys stored (never the values).
-function pullAndStore(opts = {}) {
+// Async because the service path is an HTTP call. The single caller (`bankid pull`) already awaits.
+async function pullAndStore(opts = {}) {
+  // Service first — the path with no kubectl. The k8s reader stays as the fallback so a developer
+  // mid-migration, or one whose service is unreachable, is not stuck.
+  try {
+    const fromService = await pullFromService(opts.env || 'staging')
+    if (fromService && Object.keys(fromService).length) {
+      const storedFromService = []
+      for (const [key, value] of Object.entries(fromService)) {
+        keychainSet(key, value)
+        storedFromService.push(key)
+      }
+      return storedFromService
+    }
+  } catch (_) {
+    /* fall through to the kubectl path */
+  }
+
   const values = pullFromStaging(opts)
   const stored = []
   Object.entries(values).forEach(([key, value]) => {
@@ -119,6 +145,7 @@ function loadEnv() {
 }
 
 module.exports = {
+  pullFromService,
   SECRET_KEYS,
   DEFAULT_NAMESPACE,
   DEFAULT_SECRET,

package/lib/command/bankid.js

@@ -16,8 +16,8 @@ module.exports = {
           y2
             .option('namespace', { type: 'string', default: bankid.DEFAULT_NAMESPACE })
             .option('secret', { type: 'string', default: bankid.DEFAULT_SECRET }),
-        handler: (argv) => {
-          const stored = bankid.pullAndStore({ namespace: argv.namespace, secret: argv.secret })
+        handler: async (argv) => {
+          const stored = await bankid.pullAndStore({ namespace: argv.namespace, secret: argv.secret })
           console.log(chalk.green(`bankid: stored ${stored.length} secret(s) in the keychain: ${stored.join(', ')}`))
           console.log(chalk.gray('run `dooer-test-env up` (or restart service-accounts) to apply — production BankID.'))
         },

package/lib/command/customer.js

@@ -1,38 +1,17 @@
 const fs = require('fs')
 const chalk = require('chalk')
-const engine = require('../engine/seed')
 const { purge } = require('../engine/purge')
 const { createAccount, SUBSCRIPTION_TYPES } = require('../account')
 
-// Map yargs values back to the engine's own flag argv, then let the engine's proven parser apply its
-// defaults + validation. Keeps ONE source of truth for option semantics (lib/engine/seed.js).
-function toEngineArgv(argv) {
-  const a = []
-  const flag = (name, val) => {
-    if (val !== undefined && val !== null && val !== false) a.push(`--${name}`, String(val))
-  }
-  flag('source', argv.source)
-  flag('target', argv.target)
-  flag('name', argv.name)
-  flag('owner-user', argv.ownerUser)
-  // `local` is a first-class TARGET/SOURCE, not a k8s namespace — --source-local / --target-local keep the
-  // two ideas separate rather than overloading --*-namespace (Jimmy 2026-09-03). The engine addresses the
-  // local stack through the pseudo-namespace `local` internally.
-  if (argv.sourceNamespace === 'local' || argv.targetNamespace === 'local') {
-    throw new Error('use --source-local / --target-local for the local env, not --*-namespace local')
-  }
-  flag('source-namespace', argv.sourceLocal ? 'local' : argv.sourceNamespace)
-  flag('target-namespace', argv.targetLocal ? 'local' : argv.targetNamespace)
-  flag('email', argv.email)
-  flag('if-target-nonempty', argv.ifTargetNonempty)
-  flag('salt', argv.salt)
-  if (argv.only) a.push('--only', [].concat(argv.only).join(','))
-  if (argv.skipTables) a.push('--skip-tables', [].concat(argv.skipTables).join(','))
-  if (argv.execute) a.push('--execute')
-  if (argv.skipFiles) a.push('--skip-files')
-  if (argv.skipUsers) a.push('--skip-users')
-  if (argv.confirmProduction) a.push('--confirm-production')
-  return a
+// `copy` now runs through service-dooer-test-env (export in the source, import in the target) instead of
+// opening two kubectl-sourced database connections. The COMMAND is unchanged — same name, same flags —
+// because that split is an implementation detail, not something a user should have to think about
+// (Jimmy 2026-09-04). These map the existing --*-namespace / --*-local flags onto environment names.
+const ENV_FOR_NAMESPACE = { 'dooer-staging': 'staging', 'dooer-production': 'production' }
+const envFromFlags = (local, namespace, fallback) => {
+  if (local) return 'local'
+  if (!namespace) return fallback
+  return ENV_FOR_NAMESPACE[namespace] || namespace
 }
 
 // Resolve which orgs to copy: exactly one of --source / --sourcefile. A source FILE is inherently a batch,
@@ -84,6 +63,7 @@ const copyOptions = (y) =>
     .option('skip-files', { type: 'boolean', describe: 'skip the S3 file copy' })
     .option('skip-users', { type: 'boolean', describe: 'skip copying referenced users missing from the target' })
     .option('confirm-production', { type: 'boolean', describe: 'REQUIRED to write to dooer-production' })
+    .option('reason', { type: 'string', describe: 'recorded in the audit trail at both ends' })
     .option('execute', { type: 'boolean', default: false, describe: 'actually write (default: dry-run)' })
 
 module.exports = {
@@ -97,8 +77,18 @@ module.exports = {
         builder: copyOptions,
         handler: async (argv) => {
           const sources = readSources(argv)
+          const sourceEnv = envFromFlags(argv.sourceLocal, argv.sourceNamespace, 'staging')
+          const targetEnv = envFromFlags(argv.targetLocal, argv.targetNamespace, sourceEnv)
+          const { copyOrganization } = require('../copy')
+
           if (sources.length === 1) {
-            await engine.main(engine.parseArgs(toEngineArgv({ ...argv, source: sources[0] })))
+            await copyOrganization({
+              organizationId: sources[0],
+              sourceEnv,
+              targetEnv,
+              reason: argv.reason,
+              dryRun: !argv.execute,
+            })
             return
           }
           // Batch: each org is a full independent copy (the engine scans the source per org). One bad uuid
@@ -108,7 +98,13 @@ module.exports = {
           for (let i = 0; i < sources.length; i++) {
             console.log(chalk.cyan(`\n──── [${i + 1}/${sources.length}] ${sources[i]} ────`))
             try {
-              await engine.main(engine.parseArgs(toEngineArgv({ ...argv, source: sources[i] })))
+              await copyOrganization({
+                organizationId: sources[i],
+                sourceEnv,
+                targetEnv,
+                reason: argv.reason,
+                dryRun: !argv.execute,
+              })
             } catch (e) {
               failed.push({ source: sources[i], error: e.message })
               console.log(chalk.red(`  FAILED: ${e.message}`))

package/lib/command/db.js

@@ -1,4 +1,5 @@
 const os = require('os')
+const fs = require('fs')
 const path = require('path')
 const chalk = require('chalk')
 const dbbuild = require('../engine/dbbuild')
@@ -7,6 +8,27 @@ const { shred } = require('../shred')
 const { auditPii } = require('../shred/audit')
 const { ensureRoles } = require('../db/roles')
 
+// Services whose schema exists but holds no tables beyond the migrations ledger — i.e. they were not in
+// the restored dump. Returns compose service names.
+async function servicesWithEmptySchema(local) {
+  const { Client } = require('pg')
+  const client = new Client({ ...local, ssl: false })
+  await client.connect()
+  try {
+    const { rows } = await client.query(`
+      SELECT n.nspname AS schema, count(t.table_name)::int AS tables
+      FROM information_schema.schemata n
+      LEFT JOIN information_schema.tables t
+        ON t.table_schema = n.schema_name AND t.table_name <> 'migrations'
+      WHERE n.schema_name LIKE 'service\\_%' AND n.schema_name NOT LIKE '%\\_history'
+      GROUP BY n.nspname HAVING count(t.table_name) = 0`)
+    // schema service_foo_bar -> compose service service-foo-bar
+    return rows.map((r) => r.schema.replace(/_/g, '-'))
+  } finally {
+    await client.end().catch(() => {})
+  }
+}
+
 // Base DB: the customer-free, anonymized artifact and local restore points. `db build` runs ONLY as an
 // in-cluster k8s Job (devs never run it); devs `db pull`. See ENVIRONMENT-PLAN.md §5.
 
@@ -96,8 +118,28 @@ module.exports = {
           await dbbuild.pull({
             local,
             download: async () => {
-              const cfg = obc.obcConfig({ namespace: argv.obcNamespace })
               const out = path.join(os.tmpdir(), 'dooer-base-db.pulled.dump')
+              // Service first — no kubectl, no ObjectBucketClaim credentials on this machine, no
+              // rook-ceph ClusterIP lookup. The service streams the artifact from inside the cluster.
+              try {
+                const remote = require('../remote')
+                const serviceClient = require('../service-client')
+                const env = remote.resolveEnv(argv.env || 'staging')
+                const list = await serviceClient.request(env, { path: '/v1/base-db/artifacts' })
+                const newest = (list.artifacts || [])[0]
+                if (!newest) throw new Error('no base-DB artifacts published yet')
+                console.log(`downloading ${newest.id} (${(newest.bytes / 1e6).toFixed(0)} MB) from ${env.label} …`)
+                const bytes = await serviceClient.request(env, {
+                  path: `/v1/base-db/artifacts/${encodeURIComponent(newest.id)}/content`,
+                  raw: true,
+                  timeoutMs: 1800000,
+                })
+                fs.writeFileSync(out, bytes)
+                return out
+              } catch (e) {
+                console.log(chalk.yellow(`service download unavailable (${e.message}); falling back to the OBC`))
+              }
+              const cfg = obc.obcConfig({ namespace: argv.obcNamespace })
               const { file, key } = await obc.download(cfg, out)
               console.log(chalk.gray(`downloaded ${cfg.bucket}/${key}`))
               return file
@@ -108,6 +150,21 @@ module.exports = {
             // Recreate the per-service schema grants + login roles + default search_path that staging has
             // (the dump carries no cluster-global roles). Without them unqualified queries 42P01. See db/roles.js.
             const { total, applied } = await ensureRoles({ local })
+
+            // A restored dump only contains the schemas that existed when it was BUILT. A service
+            // deployed since then (service-dooer-test-env was, today) has its tables wiped by the restore
+            // and never gets them back: `up` migrated into the old database, and nothing re-migrates
+            // afterwards — the symptom is a healthy container failing every query with
+            // `relation "…" does not exist`. Restart exactly those services so their start-up migration
+            // runs against the database that is actually there now.
+            const stale = await servicesWithEmptySchema(local)
+            if (stale.length) {
+              console.log(`re-migrating ${stale.length} service(s) missing from the dump…`)
+              const { migrateServices } = require('../db/migrate-stale')
+              for (const r of migrateServices(stale)) {
+                console.log(`  ${r.ok ? '✓' : '·'} ${r.service}: ${r.detail}`)
+              }
+            }
             console.log(chalk.gray(`db roles: ${applied.length}/${total} service roles provisioned`))
           }
         },

package/lib/command/env.js

@@ -6,6 +6,10 @@ const bankid = require('../bankid')
 const { generateCompose } = require('../compose/generate')
 const discovery = require('../discovery/client')
 
+// Every profile the generator emits (lib/compose/generate.js). `down` must name them all, or compose
+// silently leaves the profiled containers running.
+const PROFILES = ['full', 'frontend', 'hq', 'booking']
+
 // Environment lifecycle for the whole local stack. The compose file is generated from the s-e032 k8s
 // manifests (lib/compose) on first `up`; the discovery-router wires services together. Default profile is
 // `full` (everything staging runs) — compose profiles are additive, so we must name it explicitly.
@@ -91,14 +95,15 @@ module.exports = [
     command: 'stop',
     describe: 'stop the running env (preserve containers/volumes)',
     handler: () => {
-      process.exitCode = rt.dc(['stop'])
+      // Same profile trap as `down` — without the profiles, compose stops nothing.
+      process.exitCode = rt.dc(['stop'], { profile: PROFILES.join(',') })
     },
   },
   {
     command: 'start',
     describe: 'start a previously-stopped env (not recreate)',
     handler: () => {
-      process.exitCode = rt.dc(['start'])
+      process.exitCode = rt.dc(['start'], { profile: PROFILES.join(',') })
     },
   },
   {
@@ -106,7 +111,14 @@ module.exports = [
     describe: 'stop and remove the env',
     builder: (y) => y.option('volumes', { type: 'boolean', describe: 'also remove volumes (wipes the local DB)' }),
     handler: (argv) => {
-      process.exitCode = rt.dc(['down', ...(argv.volumes ? ['-v'] : [])])
+      // Compose only targets containers whose profile is active. `up` starts everything under `full` (or
+      // whichever profile was asked for), so a bare `down` removed the volumes and the network but left
+      // every profiled container running — observed 2026-09-04: 86 survivors and "Network … is still in
+      // use". Name every profile so `down` means down. --remove-orphans catches containers left behind by
+      // an earlier compose file (e.g. after a service was renamed or dropped from the manifests).
+      process.exitCode = rt.dc(['down', '--remove-orphans', ...(argv.volumes ? ['-v'] : [])], {
+        profile: PROFILES.join(','),
+      })
     },
   },
   {

package/lib/command/setup.js

@@ -34,29 +34,38 @@ module.exports = {
   handler: async (argv) => {
     console.log(chalk.bold('\ndooer-test-env setup\n'))
 
+    // No kubectl. The prerequisites are now Docker, the VPN, and an ADMIN SESSION for the environment
+    // this stack is built from — service-dooer-test-env supplies everything a kubeconfig used to.
     const dockerOk = registry.have('docker') && check('docker', ['info'])
     console.log(`${ok(dockerOk)} docker (installed + daemon running)`)
-    const kubectlOk = registry.have('kubectl')
-    console.log(`${ok(kubectlOk)} kubectl installed`)
-    // k8s reachable + access: `get ns <ns>` needs the API server (VPN) AND RBAC, so it proves both at once.
-    const nsOk = kubectlOk && check('kubectl', ['get', 'ns', argv.namespace, '--request-timeout=10s'])
-    console.log(`${ok(nsOk)} k8s cluster reachable + access to namespace ${argv.namespace} (needs VPN)`)
-    const secretsOk = nsOk && check('kubectl', ['auth', 'can-i', 'get', 'secrets', '-n', argv.namespace])
-    console.log(`${ok(secretsOk)} can read secrets (registry pull secrets + DB creds) in ${argv.namespace}`)
-    const dbOk =
-      nsOk && check('kubectl', ['get', 'svc', 'dooer-database', '-n', argv.namespace, '--request-timeout=10s'])
-    console.log(`${ok(dbOk)} can reach the dooer-database service in ${argv.namespace}`)
     if (!dockerOk) throw new Error('docker is required (install Docker + start the daemon).')
-    if (!kubectlOk) throw new Error('kubectl is required.')
-    if (!nsOk)
+
+    const remote = require('../remote')
+    const serviceClient = require('../service-client')
+    const envSpec = require('../env-spec')
+    const sourceEnv = remote.resolveEnv(argv.env || 'staging')
+
+    let describe = null
+    try {
+      describe = await serviceClient.request(sourceEnv, { path: '/v1/environment', timeoutMs: 20000 })
+    } catch (e) {
+      console.log(`${ok(false)} service-dooer-test-env in ${sourceEnv.label}`)
       throw new Error(
-        `cannot reach/access namespace ${argv.namespace} — connect the VPN and check your kubectl context/RBAC.`
+        `cannot reach service-dooer-test-env in ${sourceEnv.label}: ${e.message}\n` +
+          `  · is the VPN up? (the service is not publicly exposed)\n` +
+          `  · are you logged in? \`dooer-test-env remote-environment login --env ${sourceEnv.name} --email <admin>\`\n` +
+          `  · creating users needs an ADMIN session; BankID cannot reach one.`
       )
-    if (!secretsOk) throw new Error(`no access to read secrets in ${argv.namespace} — needed for registry + DB creds.`)
+    }
+    console.log(`${ok(true)} service-dooer-test-env reachable in ${describe.environment} (needs VPN)`)
+
+    // Fetch the environment spec — replaces parsing a local new-infrastructure checkout.
+    const spec = await envSpec.refresh(sourceEnv.name)
+    console.log(`${ok(true)} environment spec: ${spec.services.length} services, ${spec.frontends.length} frontends`)
 
     // registry login (the part that used to be manual)
     console.log('\nLogging in to the private registries…')
-    const logins = await registry.loginAll({ namespace: argv.namespace })
+    const logins = await registry.loginAll({ namespace: argv.namespace, env: sourceEnv.name })
     for (const l of logins) console.log(`  ${ok(l.ok)} ${l.host}  (${l.via})`)
     if (logins.some((l) => !l.ok)) throw new Error('one or more registry logins failed')
 
@@ -73,7 +82,7 @@ module.exports = {
     // if not already stored. Non-fatal — the env still runs (test-mode BankID) without them.
     if (bankid.status().every((s) => !s.present)) {
       try {
-        const stored = bankid.pullAndStore({ namespace: argv.namespace })
+        const stored = await bankid.pullAndStore({ namespace: argv.namespace })
         console.log(`${ok(true)} BankID: pulled ${stored.length} secret(s) from staging → keychain`)
       } catch (e) {
         console.log(`${chalk.yellow('!')} BankID: skipped (${e.message}); local login stays in test mode`)

package/lib/compose/generate.js

@@ -59,7 +59,11 @@ const GLOBAL_OVERRIDES = {
   // matching roles are created locally by `db roles` (run automatically after `db pull`). Only the password
   // is forced to the shared dev secret; a service with no manifest user falls back to `dooer` (below).
   DOOER_SQL_APPLICATION_PASSWORD: DEV.sqlPassword,
-  DOOER_SQL_MIGRATION_USER: DEV.sqlUser, // migrations run as the dooer superuser; db-migrate sets the schema itself
+  // Migrations run as the `dooer` superuser: @dooer/database's db-migrate driver issues
+  // `SET SESSION ROLE <database owner>`, which a plain service role is not permitted to do
+  // ("permission denied to set role dooer"). The driver takes the target schema from its own config
+  // (`schema: <service schema>`), so migrations still land in the SERVICE'S OWN schema, not public.
+  DOOER_SQL_MIGRATION_USER: DEV.sqlUser,
   DOOER_SQL_MIGRATION_PASSWORD: DEV.sqlPassword,
   DOOER_IS_ONEPLATFORMER: 'false',
   // clear the RDS CA baked into the image, and disable node TLS verification: @dooer/database always

package/lib/compose/manifests.js

@@ -9,6 +9,7 @@
 const fs = require('fs')
 const path = require('path')
 const yaml = require('js-yaml')
+const envSpec = require('../env-spec')
 
 // Where the s-e032 (staging) manifests live by default. Callers should pass an
 // explicit `servicesDir`; this is only the fallback for `imageFor(name)`.
@@ -68,7 +69,12 @@ function parseManifestFile(filePath) {
 
 // List every dooer service defined by a `service-*.yaml` Deployment in `dir`,
 // sorted by filename for deterministic output.
+// The spec now comes from service-dooer-test-env (lib/env-spec.js), read from the live cluster, instead
+// of a local `new-infrastructure` checkout — that is the whole point of the service. The yaml reader
+// below is kept as a fallback for anyone who still has the checkout and wants to work from it, and
+// because it is what generated every stack before this migration.
 function listServices(servicesDir) {
+  if (!servicesDir && envSpec.isCached()) return envSpec.services()
   const dir = servicesDir || DEFAULT_SERVICES_DIR
   return fs
     .readdirSync(dir)
@@ -80,6 +86,7 @@ function listServices(servicesDir) {
 
 // List every frontend defined by a `frontend-*.yaml` Deployment in `dir`.
 function listFrontends(servicesDir) {
+  if (!servicesDir && envSpec.isCached()) return envSpec.frontends()
   const dir = servicesDir || DEFAULT_SERVICES_DIR
   return fs
     .readdirSync(dir)

package/lib/copy.js

@@ -0,0 +1,101 @@
+// `customer copy`, orchestrated across two environments.
+//
+// The user-facing command does not change: one `copy`, same flags. Export-then-import is an
+// implementation detail (Jimmy 2026-09-04). What changed underneath is that neither half needs a
+// kubeconfig — each is an admin-authorized call to service-dooer-test-env in its own environment:
+//
+//    POST <source>/v1/exports    authorized by admin IN THE SOURCE   -> artifact
+//    POST <target>/v1/imports    authorized by admin IN THE TARGET   <- artifact
+//
+// Neither service ever validates the other's token, and the operator has to genuinely be an admin at
+// both ends — a stronger check than "can reach the cluster", which is what it replaces.
+const fs = require('fs')
+const os = require('os')
+const path = require('path')
+const chalk = require('chalk')
+
+const remote = require('./remote')
+const serviceClient = require('./service-client')
+
+const seconds = (ms) => `${Math.round(ms / 1000)}s`
+
+// Both halves are long-running and async by design, so progress has to be visible or a copy looks hung.
+function progress(label) {
+  const started = Date.now()
+  let lastStatus = null
+  return (op) => {
+    if (op.status === lastStatus) {
+      process.stdout.write('.')
+      return
+    }
+    lastStatus = op.status
+    process.stdout.write(`\n  ${label}: ${op.status} (${seconds(Date.now() - started)})`)
+  }
+}
+
+async function copyOrganization({ organizationId, sourceEnv, targetEnv, reason, dryRun = false }) {
+  const source = remote.resolveEnv(sourceEnv)
+  const target = remote.resolveEnv(targetEnv)
+
+  console.log(chalk.bold(`\ncopying ${organizationId}`) + `  ${source.label} → ${target.label}\n`)
+
+  // ── export ────────────────────────────────────────────────────────────────
+  const exportOp = await serviceClient.request(source, {
+    method: 'POST',
+    path: '/v1/exports',
+    body: { organizationId, reason: reason || `copy to ${target.name}` },
+  })
+  const exported = await serviceClient.pollOperation(source, 'exports', exportOp.id, {
+    onTick: progress('export'),
+  })
+  if (exported.status !== 'succeeded') {
+    throw new Error(`\nexport failed: ${exported.error || 'unknown error'}`)
+  }
+  const exportDetail = serviceClient.detailOf(exported)
+  console.log(`\n  export:  ${(exportDetail.bytes / 1e6).toFixed(1)} MB artifact`)
+
+  const skipped = exportDetail.skippedInSource || []
+  if (skipped.length) {
+    // Never let a partial copy pass silently — the whole point of recording skips service-side.
+    console.log(chalk.yellow(`  WARNING: ${skipped.length} table(s) could not be read from the source:`))
+    for (const s of skipped) console.log(chalk.yellow(`    ${s.skippedTable}: ${s.error}`))
+  }
+
+  // Straight to a file: an artifact this size should never sit in the CLI's heap.
+  const artifactPath = path.join(os.tmpdir(), `dooer-copy-${exported.id}.ndjson.gz`)
+  await serviceClient.downloadTo(source, `/v1/exports/${exported.id}/content`, artifactPath)
+
+  // ── import ────────────────────────────────────────────────────────────────
+  let importOp
+  try {
+    importOp = await serviceClient.uploadArtifactFile(
+      target,
+      `/v1/imports?${dryRun ? 'dryRun=true&' : ''}reason=${encodeURIComponent(reason || `copy from ${source.name}`)}`,
+      artifactPath
+    )
+  } finally {
+    try {
+      fs.unlinkSync(artifactPath)
+    } catch (_) {
+      /* already gone */
+    }
+  }
+  const imported = await serviceClient.pollOperation(target, 'imports', importOp.id, {
+    onTick: progress('import'),
+  })
+  if (imported.status !== 'succeeded') {
+    throw new Error(`\nimport failed: ${imported.error || 'unknown error'}`)
+  }
+  const detail = serviceClient.detailOf(imported)
+
+  console.log(
+    (dryRun ? chalk.yellow('\n\nDRY RUN — nothing written') : chalk.green('\n\ncopied')) +
+      `  ${detail.rows} rows across ${detail.tables} tables`
+  )
+  console.log(`  new organization id: ${chalk.bold(detail.organizationId)}`)
+  console.log(`  audit: export ${exported.id} (${source.name}), import ${imported.id} (${target.name})\n`)
+
+  return { organizationId: detail.organizationId, rows: detail.rows, tables: detail.tables, skipped }
+}
+
+module.exports = { copyOrganization }

package/lib/db/migrate-stale.js

@@ -0,0 +1,60 @@
+// Re-run migrations for services whose tables the base-DB restore wiped.
+//
+// A restored dump only contains the schemas that existed when it was BUILT, so a service deployed since
+// then comes up healthy with no tables and fails every query with `relation "…" does not exist`.
+// Restarting the container is supposed to fix that — the base image runs `yarn db-migrate` before
+// starting — but that wrapper cannot be relied on here: @dooer/database's `migrate` hands the real work to
+// a `resolveBin('db-migrate', …)` CALLBACK and does not await it, so the process frequently exits before
+// db-migrate has done anything. It reports success either way (exit 0, no output), which is the worst
+// combination: a container that looks migrated and is not.
+//
+// So we run db-migrate directly with the config @dooer/database builds. That config already carries the
+// right `schema`, so the tables land in the service's OWN schema.
+const { execFileSync } = require('child_process')
+
+const CONTAINER_PREFIX = 'dooer-test-env-'
+
+// Written inside the container: build the db-migrate config from the service's own env, then run it.
+const SCRIPT = `
+  set -e
+  PATH="$PATH:./node_modules/.bin"
+  node -e '
+    const fs = require("fs")
+    const db = require("@dooer/database")({
+      serviceName: process.env.DOOER_SERVICE_NAME,
+      host: process.env.DOOER_SQL_HOST,
+      port: Number(process.env.DOOER_SQL_PORT || 5432),
+      database: process.env.DOOER_SQL_DATABASE || "dooer",
+      migrationUser: process.env.DOOER_SQL_MIGRATION_USER,
+      migrationPassword: process.env.DOOER_SQL_MIGRATION_PASSWORD,
+      applicationUser: process.env.DOOER_SQL_APPLICATION_USER,
+      applicationPassword: process.env.DOOER_SQL_APPLICATION_PASSWORD,
+      isOneplatformer: process.env.DOOER_IS_ONEPLATFORMER === "true",
+    })
+    fs.writeFileSync("/tmp/db-migrate.json", JSON.stringify(db._dbMigrateConfig))
+  '
+  db-migrate --config /tmp/db-migrate.json up
+`
+
+// Returns { service, ok, detail } per service.
+function migrateServices(services) {
+  return services.map((service) => {
+    const container = `${CONTAINER_PREFIX}${service}-1`
+    try {
+      const out = execFileSync('docker', ['exec', container, 'sh', '-c', SCRIPT], {
+        encoding: 'utf8',
+        stdio: ['ignore', 'pipe', 'pipe'],
+        maxBuffer: 8 * 1024 * 1024,
+      })
+      const applied = (out.match(/Processed migration/g) || []).length
+      return { service, ok: true, detail: applied ? `${applied} migration(s)` : 'already up to date' }
+    } catch (e) {
+      // A service with no migrations directory, or one that is not running, is not a failure worth
+      // stopping a `db pull` for — report it and move on.
+      const msg = (e.stderr || e.stdout || e.message || '').toString().trim().split('\n').slice(-1)[0]
+      return { service, ok: false, detail: msg.slice(0, 120) }
+    }
+  })
+}
+
+module.exports = { migrateServices }

package/lib/db/roles.js

@@ -42,6 +42,12 @@ function schemaSetupSql(schema) {
     BEGIN TRANSACTION;
     CREATE SCHEMA IF NOT EXISTS ${ident(schema)};
     CREATE SCHEMA IF NOT EXISTS ${ident(`${schema}_history`)};
+    -- Every service keeps its OWN migrations ledger in its own schema; the base DB dump carries one per
+    -- service. A service that did not exist when the dump was built has none, so @dooer/database's
+    -- migrate resolves \`migrations\` down the search_path to the owner-only \`public.migrations\` and
+    -- fails with "permission denied for table migrations" — a brand-new service then cannot migrate
+    -- locally at all. Give it an empty ledger of its own so migrations run from scratch.
+    CREATE TABLE IF NOT EXISTS ${ident(schema)}.migrations (LIKE public.migrations INCLUDING ALL);
     DO $body$ BEGIN
       IF NOT EXISTS (SELECT FROM pg_catalog.pg_roles WHERE rolname = ${literal(`${schema}_access`)}) THEN
         CREATE ROLE ${ident(`${schema}_access`)};

package/lib/env-spec.js

@@ -0,0 +1,65 @@
+// The environment spec — what services and frontends exist, their images, ports and env.
+//
+// This used to come from a local checkout of `new-infrastructure` parsed with js-yaml, which meant every
+// developer needed that repo and a spec that silently went stale. It now comes from
+// service-dooer-test-env's `/v1/environment/compose-spec`, read from the LIVE Deployments in the cluster.
+//
+// The fetched spec is cached in the run directory so `up`, `status` and `service local` do not each make a
+// network call, and so the stack can be brought up while the VPN is down. Refresh with
+// `setup` or `up --regenerate`.
+const fs = require('fs')
+const path = require('path')
+
+const rt = require('./runtime')
+
+const SPEC_FILE = path.join(rt.RUN_DIR, 'environment-spec.json')
+
+// Fetch from the service and cache. `env` defaults to staging: the local stack is a copy of staging, so
+// that is where its shape comes from.
+async function refresh(envName = 'staging') {
+  // Required lazily: service-client reaches back into the compose generator for the local port, and a
+  // top-level require here would close the loop (manifests -> env-spec -> service-client -> generate ->
+  // manifests) and hand out half-initialised modules.
+  const remote = require('./remote')
+  const serviceClient = require('./service-client')
+  const env = remote.resolveEnv(envName)
+  const spec = await serviceClient.request(env, { path: '/v1/environment/compose-spec', timeoutMs: 120000 })
+  rt.ensureRunDir()
+  fs.writeFileSync(SPEC_FILE, JSON.stringify({ fetchedAt: new Date().toISOString(), ...spec }, null, 2))
+  return spec
+}
+
+function cached() {
+  try {
+    return JSON.parse(fs.readFileSync(SPEC_FILE, 'utf8'))
+  } catch (_) {
+    return null
+  }
+}
+
+// The generator wants `{ name, image, env, secretEnv, port }` — the same shape the yaml parser produced,
+// so nothing downstream changes.
+function readCachedOrThrow() {
+  const spec = cached()
+  if (!spec) {
+    throw new Error(
+      'no environment spec cached — run `dooer-test-env setup` (needs the VPN and an admin session for staging)'
+    )
+  }
+  return spec
+}
+
+const services = () => readCachedOrThrow().services || []
+const frontends = () => readCachedOrThrow().frontends || []
+
+function imageFor(name) {
+  const spec = cached()
+  if (!spec) return null
+  const hit = [...(spec.services || []), ...(spec.frontends || [])].find((s) => s.name === name)
+  return hit ? hit.image : null
+}
+
+const isCached = () => cached() !== null
+const fetchedAt = () => (cached() || {}).fetchedAt || null
+
+module.exports = { SPEC_FILE, refresh, cached, services, frontends, imageFor, isCached, fetchedAt }

package/lib/registry.js

@@ -20,6 +20,19 @@ function have(cmd) {
   return !r.error
 }
 
+// Pull credentials from service-dooer-test-env — the path that needs no kubectl. Returns the same
+// { host: {username, password} } shape as the kubectl reader below, so the caller cannot tell them apart.
+async function authsFromService(envName = 'staging') {
+  const remote = require('./remote')
+  const serviceClient = require('./service-client')
+  const env = remote.resolveEnv(envName)
+  const { registries } = await serviceClient.request(env, { path: '/v1/bootstrap/registry-credentials' })
+  const out = {}
+  for (const r of registries || [])
+    out[r.host.replace(/^https?:\/\//, '')] = { username: r.username, password: r.password }
+  return Object.keys(out).length ? out : null
+}
+
 // { host: { username, password } } parsed from a k8s dockerconfigjson secret (or null if unreadable)
 function authsFromSecret(secretName, namespace) {
   let b64
@@ -77,6 +90,21 @@ async function prompt(question, { silent } = {}) {
 
 // Log docker into both private registries. Returns [{host, ok, via}]. `via` = kubectl | env | prompt.
 async function loginAll({ namespace = 'dooer-staging', interactive = true } = {}) {
+  // Service first — that is the path with no kubectl. Falls back to the Secret reader and then to
+  // env/prompt, so a developer mid-migration (or with the service unreachable) is never stuck.
+  try {
+    const fromService = await authsFromService()
+    if (fromService) {
+      const results = []
+      for (const [host, creds] of Object.entries(fromService)) {
+        results.push({ host, ok: dockerLogin(host, creds.username, creds.password), via: 'service' })
+      }
+      if (results.length && results.every((r) => r.ok)) return results
+    }
+  } catch (_) {
+    /* fall through to the kubectl path below */
+  }
+
   if (!have('docker')) throw new Error('docker is not installed / not on PATH')
   const results = []
   const merged = {}

package/lib/remote.js

@@ -113,12 +113,18 @@ function tokenFor(envName) {
   if (stored) {
     const exp = expiry(stored)
     if (exp && exp.expired) {
-      throw new Error(
-        `your ${envName} session expired ${exp.at.toISOString().slice(0, 16).replace('T', ' ')} — ` +
-          `run: dooer-test-env remote-environment login --env ${envName}`
-      )
+      // The local stack needs no login at all, so an expired session there is not a reason to stop —
+      // fall through and mint. Blocking on it turned a stale keychain entry into a hard failure in the
+      // middle of a copy.
+      if (envName !== 'local') {
+        throw new Error(
+          `your ${envName} session expired ${exp.at.toISOString().slice(0, 16).replace('T', ' ')} — ` +
+            `run: dooer-test-env remote-environment login --env ${envName}`
+        )
+      }
+    } else {
+      return stored
     }
-    return stored
   }
   if (envName === 'local') return require('./api').serviceToken()
   throw new Error(`not logged in to ${envName} — run: dooer-test-env remote-environment login --env ${envName}`)

package/lib/runtime.js

@@ -12,9 +12,12 @@ const OVERRIDE_FILE = path.join(RUN_DIR, 'docker-compose.override.yml')
 const STATE_FILE = path.join(RUN_DIR, 'state.json')
 const REGISTRY_FILE = path.join(RUN_DIR, 'registry.json')
 const ROUTER_URL = process.env.DOOER_TEST_ENV_ROUTER || 'http://localhost:8500'
-const SERVICES_DIR =
-  process.env.DOOER_TEST_ENV_MANIFESTS ||
-  path.join(os.homedir(), 'dooer', 'new-infrastructure', 'kubernetes', 'environments', 's-e032-onprem')
+// The environment spec now comes from service-dooer-test-env, cached by lib/env-spec.js — no
+// `new-infrastructure` checkout required. This stays as an explicit ESCAPE HATCH: set
+// DOOER_TEST_ENV_MANIFESTS to a manifest directory to generate from local yaml instead, which is useful
+// when working on manifests that are not deployed yet. Undefined by default, so callers passing it
+// through fall to the cached spec.
+const SERVICES_DIR = process.env.DOOER_TEST_ENV_MANIFESTS || undefined
 
 function ensureRunDir() {
   fs.mkdirSync(RUN_DIR, { recursive: true })

package/lib/service-client.js

@@ -0,0 +1,161 @@
+/* global fetch, AbortSignal, FormData */
+// Talking to service-dooer-test-env — the service that replaced this CLI's kubectl access.
+//
+// It is deliberately NOT publicly exposed (no ingress, no public-facade route), so it is reached the way
+// anything in the cluster is reached from a laptop: over the WireGuard tunnel, at the ClusterIP of its
+// k8s Service object. Those IPs are shipped as defaults; they are stable because the Service objects are
+// never deleted (Jimmy 2026-09-04), and DNS may replace them later.
+//
+// The LOCAL stack runs its own copy of the service — it appears in the compose spec like every other
+// staging Deployment — reached on its published host port.
+const fs = require('fs')
+const { Readable } = require('stream')
+const { pipeline: streamPipeline } = require('stream/promises')
+
+const remote = require('./remote')
+const rt = require('./runtime')
+
+const SERVICE_NAME = 'service-dooer-test-env'
+
+// Per-environment endpoints. Overridable so nobody is stuck if an IP changes before the default does.
+const CLUSTER_ENDPOINTS = {
+  staging: process.env.DOOER_TEST_ENV_SERVICE_STAGING || 'http://10.108.228.242',
+  production: process.env.DOOER_TEST_ENV_SERVICE_PRODUCTION || 'http://10.109.163.108',
+}
+
+function endpointFor(envName) {
+  if (envName === 'local') {
+    // Lazy: the generator requires the manifest reader, which requires env-spec, which requires this
+    // module. Deferring the require keeps that cycle from forming at load time.
+    const { serviceHostPortMap } = require('./compose/generate')
+    const { listServices } = require('./compose/manifests')
+    const port = serviceHostPortMap(listServices(rt.SERVICES_DIR))[SERVICE_NAME]
+    if (!port) {
+      throw new Error(
+        `${SERVICE_NAME} is not in the local stack — run \`dooer-test-env up --regenerate\` so the ` +
+          `compose file picks it up`
+      )
+    }
+    return `http://localhost:${port}`
+  }
+  const url = CLUSTER_ENDPOINTS[envName]
+  if (!url) {
+    throw new Error(
+      `no ${SERVICE_NAME} endpoint known for "${envName}" — it may not be deployed there yet. ` +
+        `Set DOOER_TEST_ENV_SERVICE_${envName.toUpperCase()} to override.`
+    )
+  }
+  return url
+}
+
+// One request. `env` is a resolved environment from lib/remote.
+async function request(env, { method = 'GET', path, body, raw = false, timeoutMs = 120000 }) {
+  const url = `${endpointFor(env.name)}${path}`
+  const headers = { authorization: `Bearer ${remote.tokenFor(env.name)}`, 'x-dooer-client': 'dooer-test-env@0' }
+  if (body !== undefined) headers['content-type'] = 'application/json'
+  let res
+  try {
+    res = await fetch(url, {
+      method,
+      headers,
+      body: body === undefined ? undefined : JSON.stringify(body),
+      signal: AbortSignal.timeout(timeoutMs),
+    })
+  } catch (e) {
+    throw new Error(
+      `cannot reach ${SERVICE_NAME} for ${env.label} at ${url}: ${e.message}` +
+        (env.name === 'local' ? '' : ' — is the VPN up?')
+    )
+  }
+  if (raw) {
+    if (!res.ok) throw new Error(`${env.label}: HTTP ${res.status} from ${path}`)
+    return Buffer.from(await res.arrayBuffer())
+  }
+  const text = await res.text()
+  let json
+  try {
+    json = JSON.parse(text)
+  } catch (_) {
+    throw new Error(`${env.label}: HTTP ${res.status} from ${path} (not JSON): ${text.slice(0, 200)}`)
+  }
+  if (!res.ok) {
+    const detail = json.detail || json.title || json.code || `HTTP ${res.status}`
+    throw new Error(`${env.label}: ${detail}`)
+  }
+  return json
+}
+
+// Download a route's body straight to a FILE. A real organization's artifact is hundreds of megabytes —
+// Ghost Inspector is 398 MB gzipped — and holding it in a Buffer (then again inside a Blob) is a needless
+// way to run a laptop out of memory.
+async function downloadTo(env, routePath, filePath, { timeoutMs = 1800000 } = {}) {
+  const url = `${endpointFor(env.name)}${routePath}`
+  const res = await fetch(url, {
+    headers: { authorization: `Bearer ${remote.tokenFor(env.name)}`, 'x-dooer-client': 'dooer-test-env@0' },
+    signal: AbortSignal.timeout(timeoutMs),
+  })
+  if (!res.ok) throw new Error(`${env.label}: HTTP ${res.status} downloading ${routePath}`)
+  await streamPipeline(Readable.fromWeb(res.body), fs.createWriteStream(filePath))
+  return fs.statSync(filePath).size
+}
+
+// Upload the artifact as multipart — the fleet's way of moving binary through a route. Sourced from a
+// file so it is streamed rather than materialised.
+async function uploadArtifactFile(env, path, filePath, { timeoutMs = 1800000 } = {}) {
+  const form = new FormData()
+  form.append('artifact', await fs.openAsBlob(filePath, { type: 'application/x-ndjson+gzip' }), 'artifact.ndjson.gz')
+  const url = `${endpointFor(env.name)}${path}`
+  const res = await fetch(url, {
+    method: 'POST',
+    headers: { authorization: `Bearer ${remote.tokenFor(env.name)}`, 'x-dooer-client': 'dooer-test-env@0' },
+    body: form,
+    signal: AbortSignal.timeout(timeoutMs),
+  })
+  const text = await res.text()
+  let json
+  try {
+    json = JSON.parse(text)
+  } catch (_) {
+    throw new Error(`${env.label}: HTTP ${res.status} on import (not JSON): ${text.slice(0, 200)}`)
+  }
+  if (!res.ok) throw new Error(`${env.label}: ${json.detail || json.code || `HTTP ${res.status}`}`)
+  return json
+}
+
+// Operations are async by design — a full organization copy runs for minutes, and a request that long is
+// the wrong shape. Poll until it settles, reporting progress so a long wait does not look like a hang.
+async function pollOperation(env, kind, id, { onTick, intervalMs = 5000, timeoutMs = 3600000 } = {}) {
+  const deadline = Date.now() + timeoutMs
+  for (;;) {
+    const op = await request(env, { path: `/v1/${kind}/${id}` })
+    if (onTick) onTick(op)
+    if (op.status !== 'running') return op
+    if (Date.now() > deadline) {
+      throw new Error(`${kind} ${id} still running after ${Math.round(timeoutMs / 60000)} minutes — giving up waiting`)
+    }
+    await new Promise((r) => setTimeout(r, intervalMs))
+  }
+}
+
+// jsonb comes back off the model as a string; callers want the object.
+function detailOf(operation) {
+  const d = operation && operation.detail
+  if (!d) return {}
+  if (typeof d !== 'string') return d
+  try {
+    return JSON.parse(d)
+  } catch (_) {
+    return {}
+  }
+}
+
+module.exports = {
+  SERVICE_NAME,
+  endpointFor,
+  request,
+  downloadTo,
+  uploadArtifactFile,
+  pollOperation,
+  detailOf,
+  CLUSTER_ENDPOINTS,
+}

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@dooer/dooer-test-env",
-  "version": "1.16.0",
+  "version": "1.17.0",
   "description": "Run the whole Dooer backend locally (staging DB minus customers), copy/purge customers between environments, and shred — one CLI.",
   "license": "UNLICENSED",
   "repository": "Dooer/cli-dooer-test-env",

package/readme.md

@@ -6,7 +6,8 @@ organizations). Copy specific customers in when you need them, spin up empty tes
 across the whole stack, and measure resource usage.
 
 Runs on an Apple-Silicon Mac (images run under emulation) — 32 GB RAM / ~150 GB free disk is comfortable.
-See **[ENVIRONMENT-PLAN.md](./ENVIRONMENT-PLAN.md)** for the full design and rationale.
+See **[ENVIRONMENT-PLAN.md](./ENVIRONMENT-PLAN.md)** for the full design and rationale, and
+**[REVERT-POINT.md](./REVERT-POINT.md)** for the last version that works without `service-dooer-test-env`.
 
 ```bash
 npx @dooer/dooer-test-env <command>
@@ -14,24 +15,32 @@ npx @dooer/dooer-test-env <command>
 
 ## First time here?
 
-`setup` needs **VPN access** and a **kubectl credential** for the cluster — the compose file, the registry
-logins and the BankID cert all come from there.
+You need exactly two things:
 
-- **[Getting access](./docs/getting-access.md)** — new developer: set up the VPN, get your kubectl
-  certificate, and verify both actually work.
-- **[Issuing access](./docs/issuing-access.md)** — admin: sign someone's certificate, bind their
-  permissions, and revoke them again.
-- **[Logins and user management](./docs/user-management.md)** — signing in to an environment as a customer,
-  partner or admin user; creating users; partner membership; using the token from scripts. Needs no VPN or
-  kubectl at all.
+1. **The VPN** — everything this tool talks to lives inside the cluster network.
+2. **An admin session for staging** — `remote-environment login`. That is what authorizes the tool to
+   fetch the environment spec, the registry credentials and the BankID material.
 
-Already have both? Straight to the quick start.
+**No kubectl. No cluster credential. No `new-infrastructure` checkout.** All of that moved behind
+`service-dooer-test-env`, which runs in each environment and exposes only the operations this tool needs
+— each one admin-gated and recorded. See
+[the specification](./_specification/service-dooer-test-env.md) for what it replaced and why.
+
+- **[Getting access](./docs/getting-access.md)** — new developer: set up the VPN and log in.
+- **[Logins and user management](./docs/user-management.md)** — signing in as a customer, partner or admin
+  user; creating users; partner membership; using the token from scripts.
+- **[Issuing access](./docs/issuing-access.md)** — admin: VPN tunnels, and cluster certificates for the
+  people who actually operate the cluster. **Not needed to use this tool any more.**
 
 ## Quick start
 
 ```bash
-# 1. One-time: prereq checks, registry login, compose generation, BankID cert → keychain.
-#    Needs the VPN + kubectl access to dooer-staging (see "First time here?" above).
+# 0. Log in to staging as an ADMIN — the local stack is built from staging's live Deployments.
+#    BankID cannot reach an admin account; this has to be email + password.
+npx @dooer/dooer-test-env remote-environment login --env staging --email you+admin@dooer.com
+
+# 1. One-time: prereqs, registry login, environment spec, compose generation, BankID cert → keychain.
+#    Needs the VPN and the admin session above.
 npx @dooer/dooer-test-env setup
 
 # 2. Bring the whole stack up (all staging services + frontends)
@@ -214,9 +223,23 @@ The local env needs no login for this — with no stored session it mints one fr
 
 ### Copy a real customer into the local env
 
-`--target-local` addresses this stack (Postgres on 55432 + MinIO); the source can be any k8s namespace
-(`--source-local` goes the other way). Reading production needs no confirmation — only *writing* to it does. Emails are always
-anonymized. Dry-run by default; add `--execute`.
+`--target-local` addresses this stack; the source can be any environment (`--source-local` goes the other
+way). Emails are always anonymized. Dry-run by default; add `--execute`.
+
+**You need an admin session in BOTH ends.** A copy is two locally-authorized halves — an export authorized
+by admin in the *source* and an import authorized by admin in the *target* — because each environment
+signs its own tokens and neither service will ever validate the other's. That is a stronger check than the
+kubeconfig it replaces, where reaching the cluster was enough. The command is unchanged; the split is
+internal.
+
+```bash
+npx @dooer/dooer-test-env@latest remote-environment login --env production --email you+admin@dooer.com
+npx @dooer/dooer-test-env@latest remote-environment login --env staging --email you+admin@dooer.com
+```
+
+Both halves are async, so the command prints progress while it waits. If the source could not read a
+table, the copy says so as a **WARNING** rather than quietly producing a partial org, and both ends record
+the operation in their audit trail (`remote-environment` → the service's `/v1/operations`).
 
 ```bash
 # copy Ghost Inspector out of LIVE into the local env, as a new org owned by a local user
@@ -379,6 +402,10 @@ Inbound *input* validation always runs — this only affects response validation
 
 ## Prerequisites
 
-Docker (Desktop), `kubectl` with access to the `dooer-staging` namespace (VPN), and Node 18+.
+Docker (Desktop), the **VPN**, an **admin session for staging**, and Node 18+.
+
+No kubectl, no cluster certificate, no `new-infrastructure` checkout — `service-dooer-test-env` supplies
+what those used to. Don't have the VPN yet? → **[Getting access](./docs/getting-access.md)**.
 
-Don't have the VPN or a kubectl credential yet? → **[Getting access](./docs/getting-access.md)**.
+`DOOER_TEST_ENV_MANIFESTS=<dir>` still generates from local manifest yaml instead of the live cluster,
+which is useful when you are working on a manifest that is not deployed yet.