hazo_env 0.9.0 → 0.10.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGE_LOG.md +50 -0
- package/README.md +14 -3
- package/config/hazo_env_config.ini.sample +42 -0
- package/dist/cli.js +31 -8
- package/dist/envsync/dotenv-merge.d.ts +46 -3
- package/dist/envsync/dotenv-merge.d.ts.map +1 -1
- package/dist/envsync/dotenv-merge.js +94 -8
- package/dist/envsync/engine.d.ts +18 -0
- package/dist/envsync/engine.d.ts.map +1 -1
- package/dist/envsync/engine.js +63 -26
- package/dist/envsync/guards.d.ts +22 -2
- package/dist/envsync/guards.d.ts.map +1 -1
- package/dist/envsync/guards.js +41 -4
- package/dist/envsync/paths.d.ts +10 -0
- package/dist/envsync/paths.d.ts.map +1 -0
- package/dist/envsync/paths.js +67 -0
- package/dist/envsync/redact.d.ts +14 -0
- package/dist/envsync/redact.d.ts.map +1 -0
- package/dist/envsync/redact.js +41 -0
- package/dist/envsync/service.d.ts.map +1 -1
- package/dist/envsync/service.js +22 -7
- package/dist/resolve/envsync.d.ts +19 -4
- package/dist/resolve/envsync.d.ts.map +1 -1
- package/dist/resolve/envsync.js +14 -1
- package/package.json +12 -12
package/CHANGE_LOG.md
CHANGED
|
@@ -1,5 +1,55 @@
|
|
|
1
1
|
# hazo_env — Change Log
|
|
2
2
|
|
|
3
|
+
## 0.10.0 — 2026-07-29
|
|
4
|
+
|
|
5
|
+
Security/safety hardening of `envsync`. Six defects fixed, each with a regression test. These are **behavior changes** — new validation refuses inputs that previously ran, one new opt-in flag is required to keep an old behavior, and diff/log output is now redacted.
|
|
6
|
+
|
|
7
|
+
### Fixed — destructive-before-validate
|
|
8
|
+
- **`uploadFiles` no longer wipes `files_root` before checking the archive.** The archive is now validated first — path resolves inside `work_dir`, is a non-empty regular file, and lists cleanly under `tar -tzf` — and only then is `files_root` removed. Worst case before: `upload-files --archive <typo> --confirm --skip-env` deleted the entire destination and *then* failed, because under `--skip-env` nothing else touched the archive before the wipe.
|
|
9
|
+
- **`uploadDb` no longer runs `pre_restore_cmd`/`dropdb` before checking the dump.** `dumpPath` is validated (exists, non-empty, regular file, inside `work_dir`) immediately after the confirm/prod guards, before anything destructive. A typo'd `--dump` used to drop the target database and only then discover the file wasn't there.
|
|
10
|
+
- Validation lives in the new `src/envsync/paths.ts::resolveArtifactPath()`. All its errors start with `Refusing`, so the HTTP service maps them to `400`.
|
|
11
|
+
|
|
12
|
+
### Fixed — argument injection / path confinement
|
|
13
|
+
- `pg_restore` is now invoked as `pg_restore -d <db> -- <dump>`; the `--` end-of-options separator means a path can no longer be read as a flag.
|
|
14
|
+
- `dumpPath` / `archivePath` are rejected outright when they start with `-` (e.g. `--jobs=8`, `--checkpoint-action=exec=…`) or contain a NUL byte.
|
|
15
|
+
- Both paths must resolve **inside `work_dir`**, checked against the real (symlink-resolved) path — so `..` traversal and a symlink inside `work_dir` pointing elsewhere both fail. The resolved path is what gets passed to the child process, which also closes the check-then-use symlink-swap window.
|
|
16
|
+
|
|
17
|
+
### Fixed — `.env.local` serialization was lossy
|
|
18
|
+
- `serializeDotenv` wrote bare `KEY=value`, silently corrupting any value containing a space, `#`, quote, `=`, or newline — i.e. every PEM key, JSON blob and passphrase. It now picks a quoting form that `dotenv.parse` restores exactly: bare for a conservative safe charset, single quotes (fully literal in dotenv) otherwise, backticks when the value contains `'`, and double quotes with `\n`/`\r` escapes when a carriage return is involved. A value that mixes all three quote characters cannot be represented in dotenv's grammar and now throws (message starts with `Refusing`) instead of corrupting.
|
|
19
|
+
- The merged env is serialized **before** `files_root` is wiped, so an unencodable value fails during the dry run, never mid-restore.
|
|
20
|
+
- `__tests__/envsync-dotenv-merge.test.ts` adds a property-style round-trip suite (`parse(serialize(x)) === x`) over ~20 hostile values.
|
|
21
|
+
|
|
22
|
+
### Fixed — silent `.env.local` key deletion
|
|
23
|
+
- `uploadFiles` used to overwrite the target `.env.local` with only the merged keys, while the dry-run diff iterated `Object.keys(merged)` — so a key that existed **only** in the target was deleted without ever appearing in the safety preview.
|
|
24
|
+
- **Target-only keys are now preserved by default.** Deleting them requires the explicit opt-in `--prune-env` (`uploadFiles({ pruneEnv: true })`, `{"pruneEnv": true}` over HTTP).
|
|
25
|
+
- The diff now covers the union of both key sets and every entry carries an explicit `status: 'added' | 'changed' | 'unchanged' | 'removed'`, so a removal can never read as a no-op.
|
|
26
|
+
- New `resolveFinalEnv(merged, current, { pruneMissing })` in `dotenv-merge.ts` decides what is actually written.
|
|
27
|
+
|
|
28
|
+
### Fixed — secrets in logs and HTTP responses
|
|
29
|
+
- New `src/envsync/redact.ts`. Every engine progress line — including `pg_dump`/`pg_restore`/`tar` stderr echoed back through `onProgress` — is passed through `redactConnectionString()`, which rewrites `scheme://user:pass@host` to `scheme://user:***@host` and redacts `password=` in query-string and libpq keyword form. `source_db`/`target_db` are commonly full `postgres://user:pass@host` URIs and were being interpolated straight into `console.log`.
|
|
30
|
+
- The `.env.local` diff — a complete plaintext dump of the target environment's secrets — is no longer returned by the HTTP API. `POST /files/restore` now responds with `{ ok, diff: [{ key, status }] }` via the new `redactEnvDiff()`; values never leave the process.
|
|
31
|
+
- The CLI hides values too, printing `key + status`. Pass `--show-values` to print them (for a local operator eyeballing a change).
|
|
32
|
+
|
|
33
|
+
### Fixed — the production guard barely guarded
|
|
34
|
+
- `prodDbNames` defaulted to `[source_db]`, so `assertNotProd` only ever fired on an exact source==target match — a staging→prod push was never caught. `assertNotProd` now also applies a **name pattern** (`DEFAULT_PROD_NAME_PATTERN`: a `prod`/`production`/`live` token anywhere in the target name, token-boundary anchored so `product_catalog_dev` does not match). `looksLikeProd()` is exported for non-throwing checks.
|
|
35
|
+
- Both halves are configurable from `[envsync]`: `prod_db_names` (comma-separated extra exact names; `source_db` is always included) and `prod_db_pattern` (regex source; empty value = exact-name list only, malformed value falls back to the default rather than widening the guard).
|
|
36
|
+
- **`allowProd` is no longer taken from the HTTP request body.** The service reads it exclusively from `cfg.allowProd` (`allow_prod = true` in `[envsync]`) — a caller can no longer hand itself the override it is being guarded by. The `serve` control page's "allow prod" checkboxes are gone.
|
|
37
|
+
|
|
38
|
+
### API changes
|
|
39
|
+
- `EnvOverrideDiff` gained a required `status` field and `after` widened to `string | undefined` (it is `undefined` for a removal). Consumers that destructured `{ key, before, after }` still work; anything doing an exact object comparison needs updating.
|
|
40
|
+
- `uploadFiles(cfg, archivePath, opts)` gained `opts.pruneEnv?: boolean`.
|
|
41
|
+
- `assertNotProd(targetName, prodNames, allowProd)` gained an optional 4th parameter `pattern?: string`. Existing 3-arg calls are unaffected.
|
|
42
|
+
- `EnvsyncConfig` gained `prodDbPattern?: string` and `allowProd?: boolean` (both optional — existing config literals still type-check).
|
|
43
|
+
- `POST /files/restore` response shape changed from `{ ok, diff: EnvOverrideDiff[] }` to `{ ok, diff: RedactedEnvDiff[] }` (`{ key, status }` only). It also accepts `pruneEnv` and now **ignores** `allowProd` in the body.
|
|
44
|
+
- New modules: `src/envsync/paths.ts` (`resolveArtifactPath`), `src/envsync/redact.ts` (`redactConnectionString`, `redactingProgress`). New exports from `dotenv-merge.ts`: `resolveFinalEnv`, `redactEnvDiff`, `EnvDiffStatus`, `RedactedEnvDiff`. New export from `guards.ts`: `looksLikeProd`, `DEFAULT_PROD_NAME_PATTERN`.
|
|
45
|
+
- New CLI flags: `hazo-env sync upload-files --prune-env`, `--show-values`.
|
|
46
|
+
|
|
47
|
+
### Config
|
|
48
|
+
- `config/hazo_env_config.ini.sample` now documents the `[envsync]` section, including `prod_db_names`, `prod_db_pattern` and `allow_prod`.
|
|
49
|
+
|
|
50
|
+
### Tests
|
|
51
|
+
- 102 → 191. New files: `__tests__/envsync-paths.test.ts`, `__tests__/envsync-redact.test.ts`, `__tests__/envsync-service-prod-guard.test.ts` (kept separate from `envsync-service.test.ts`, which mocks the engine module registry file-wide and would bypass the real guards).
|
|
52
|
+
|
|
3
53
|
## 0.9.0 — 2026-07-24
|
|
4
54
|
|
|
5
55
|
### Added
|
package/README.md
CHANGED
|
@@ -158,7 +158,8 @@ const report = await doctor({ probe: true, all: true });
|
|
|
158
158
|
`envsync` copies a database or files root between environments that live on the same host (or where you're willing to move a dump/archive by hand) using plain `pg_dump`/`pg_restore`/`tar` — no SSH, no PostgREST row-copy loop, no masking pass. It's driven via the CLI or an optional local HTTP service; the engine functions (`downloadDb`, `uploadDb`, `downloadFiles`, `uploadFiles`) live at `hazo_env/dist/envsync/engine.js` and are intentionally **not** re-exported from the package's main `index.ts` — reach them through that subpath or through the CLI.
|
|
159
159
|
|
|
160
160
|
```bash
|
|
161
|
-
# Config: [envsync] section in hazo_env_config.ini — source_db, target_db, files_root, work_dir,
|
|
161
|
+
# Config: [envsync] section in hazo_env_config.ini — source_db, target_db, files_root, work_dir,
|
|
162
|
+
# keep, prod_db_names, prod_db_pattern, allow_prod
|
|
162
163
|
|
|
163
164
|
hazo-env sync download-db # pg_dump source_db → work_dir
|
|
164
165
|
hazo-env sync upload-db --dump <path> --confirm # pg_restore into target_db (--allow-prod-target for a prod target)
|
|
@@ -166,11 +167,20 @@ hazo-env sync download-files # tar files_root (+ current
|
|
|
166
167
|
hazo-env sync upload-files --archive <path> # dry-run: prints the .env.local diff it would apply
|
|
167
168
|
hazo-env sync upload-files --archive <path> --confirm # actually restores files + merges env overrides
|
|
168
169
|
hazo-env sync upload-files --archive <path> --confirm --skip-env # restore files only; leave .env.local untouched
|
|
170
|
+
hazo-env sync upload-files --archive <path> --confirm --prune-env # also DELETE .env.local keys the archive lacks
|
|
171
|
+
hazo-env sync upload-files --archive <path> --show-values # print env values in the diff (hidden by default)
|
|
169
172
|
|
|
170
173
|
hazo-env serve [--port <n>] [--bind <host>] # HTTP service over the same engine — requires HAZO_ENVSYNC_TOKEN
|
|
171
174
|
```
|
|
172
175
|
|
|
173
|
-
|
|
176
|
+
Safety model (see `CHANGE_LOG.md` 0.7.0 for the original design and 0.10.0 for the hardening pass):
|
|
177
|
+
|
|
178
|
+
- **Single writer.** Every destructive op runs under `withLock`.
|
|
179
|
+
- **Validate, then destroy.** The dump/archive is fully checked (exists, non-empty, inside `work_dir`, and — for archives — lists cleanly under `tar -t`) before `dropdb` or the `files_root` wipe runs. A typo'd path can never delete anything.
|
|
180
|
+
- **Confined paths.** `--dump`/`--archive` must resolve (symlinks included) inside `work_dir`, must not start with `-`, and are passed after a `--` end-of-options separator.
|
|
181
|
+
- **Production guard.** A target is refused if it is on `prod_db_names` **or** matches `prod_db_pattern` (default: a `prod`/`production`/`live` token anywhere in the name). Override with `--allow-prod-target` on the CLI, or `allow_prod = true` in `[envsync]`. The HTTP service reads the override **only** from config — a request body can never set it.
|
|
182
|
+
- **`.env.local` keys are preserved.** Keys that exist only in the target survive a restore by default; deleting them requires `--prune-env` / `pruneEnv: true`. The diff preview lists every key with an `added`/`changed`/`unchanged`/`removed` status.
|
|
183
|
+
- **No secrets in output.** Connection-string credentials are stripped from every progress line, and env diffs are reported as key + status — the HTTP API and control page never carry `.env.local` values.
|
|
174
184
|
|
|
175
185
|
## CLI
|
|
176
186
|
|
|
@@ -179,7 +189,8 @@ hazo-env current # prints env, role, pattern, app, data_ro
|
|
|
179
189
|
hazo-env doctor [--env <e>] [--all] # red/green validation table
|
|
180
190
|
|
|
181
191
|
hazo-env sync download-db | upload-db --dump <p> [--confirm] [--allow-prod-target]
|
|
182
|
-
hazo-env sync download-files | upload-files --archive <p> [--confirm] [--allow-prod-target]
|
|
192
|
+
hazo-env sync download-files | upload-files --archive <p> [--confirm] [--allow-prod-target]
|
|
193
|
+
[--skip-env] [--prune-env] [--show-values]
|
|
183
194
|
hazo-env serve [--port <n>] [--bind <host>] # requires HAZO_ENVSYNC_TOKEN
|
|
184
195
|
```
|
|
185
196
|
|
|
@@ -63,6 +63,48 @@ location = remote
|
|
|
63
63
|
[host.prod]
|
|
64
64
|
location = remote
|
|
65
65
|
|
|
66
|
+
; ─── envsync: local pg_dump/pg_restore/tar sync ──────────────────────────────
|
|
67
|
+
; Drives `hazo-env sync ...` and `hazo-env serve`. Omit the whole section to
|
|
68
|
+
; disable envsync (every command then reports "not configured").
|
|
69
|
+
|
|
70
|
+
[envsync]
|
|
71
|
+
; Database pulled FROM by download-db. May be a bare name or a full
|
|
72
|
+
; postgres://user:pass@host/db URI — credentials are redacted from all logs.
|
|
73
|
+
source_db = ${ENVSYNC_SOURCE_DB}
|
|
74
|
+
; Database dropped/recreated/restored INTO by upload-db.
|
|
75
|
+
target_db = myapp_dev
|
|
76
|
+
; Owner passed to `createdb -O`.
|
|
77
|
+
owner = appuser
|
|
78
|
+
; Files root archived by download-files and WIPED+repopulated by upload-files.
|
|
79
|
+
files_root = ${DATA_ROOT}/files
|
|
80
|
+
; Scratch dir for dumps/archives + the .envsync.lock file. Dump/archive paths
|
|
81
|
+
; passed to upload-db/upload-files must resolve inside this directory.
|
|
82
|
+
work_dir = ${DATA_ROOT}/envsync
|
|
83
|
+
; How many dumps and how many archives to retain in work_dir (default 3).
|
|
84
|
+
keep = 3
|
|
85
|
+
; Optional shell commands run around the restore.
|
|
86
|
+
; pre_restore_cmd = systemctl stop myapp
|
|
87
|
+
; post_restore_cmd = systemctl start myapp
|
|
88
|
+
|
|
89
|
+
; ─── envsync production guard ────────────────────────────────────────────────
|
|
90
|
+
; A target is refused unless the operator explicitly overrides, when it is
|
|
91
|
+
; listed in prod_db_names OR matches prod_db_pattern.
|
|
92
|
+
;
|
|
93
|
+
; Extra exact names to guard, comma-separated. source_db is ALWAYS guarded and
|
|
94
|
+
; does not need listing.
|
|
95
|
+
; prod_db_names = myapp_prod, myapp_prod_replica
|
|
96
|
+
;
|
|
97
|
+
; Case-insensitive regex matched against the target name. Default catches a
|
|
98
|
+
; prod/production/live token anywhere in the name (myapp_prod, prod-myapp,
|
|
99
|
+
; myapp-live). Set it to an empty value to guard on prod_db_names alone.
|
|
100
|
+
; prod_db_pattern = (^|[^a-z0-9])(prod|production|live)([^a-z0-9]|$)
|
|
101
|
+
;
|
|
102
|
+
; Server-side override for the guard. This is the ONLY way the HTTP service
|
|
103
|
+
; (`hazo-env serve`) will ever write to a production-looking target — the
|
|
104
|
+
; request body cannot set it. Leave false unless this host is meant to push
|
|
105
|
+
; to production.
|
|
106
|
+
allow_prod = false
|
|
107
|
+
|
|
66
108
|
; ─── SSH transport for cross-host file rsync ─────────────────────────────────
|
|
67
109
|
; When [transport.ssh.<env>] is present, copyFiles will use rsync-over-SSH
|
|
68
110
|
; to pull files from that environment's file server instead of local copy.
|
package/dist/cli.js
CHANGED
|
@@ -58,7 +58,19 @@ async function runDoctor() {
|
|
|
58
58
|
process.exit(1);
|
|
59
59
|
}
|
|
60
60
|
}
|
|
61
|
-
|
|
61
|
+
const DIFF_MARKERS = {
|
|
62
|
+
added: pc.green('+'),
|
|
63
|
+
changed: pc.yellow('~'),
|
|
64
|
+
unchanged: pc.dim('='),
|
|
65
|
+
removed: pc.red('-'),
|
|
66
|
+
};
|
|
67
|
+
/**
|
|
68
|
+
* Print the .env.local preview. Values are REDACTED by default — this output
|
|
69
|
+
* routinely ends up in a scrollback buffer or a CI log, and the diff is a
|
|
70
|
+
* complete dump of the target environment's secrets. `--show-values` opts
|
|
71
|
+
* back in for a local operator who needs to eyeball the actual change.
|
|
72
|
+
*/
|
|
73
|
+
function printEnvOverrideDiff(diff, showValues) {
|
|
62
74
|
if (!diff.length) {
|
|
63
75
|
console.log(` ${pc.dim('(no env vars to apply)')}`);
|
|
64
76
|
return;
|
|
@@ -66,9 +78,14 @@ function printEnvOverrideDiff(diff) {
|
|
|
66
78
|
console.log(` ${pc.bold('.env.local changes:')}`);
|
|
67
79
|
const keyWidth = Math.max(...diff.map((d) => d.key.length)) + 2;
|
|
68
80
|
for (const d of diff) {
|
|
69
|
-
const marker = d.
|
|
70
|
-
const
|
|
71
|
-
|
|
81
|
+
const marker = DIFF_MARKERS[d.status];
|
|
82
|
+
const detail = showValues
|
|
83
|
+
? `${d.before ?? pc.dim('(unset)')} -> ${d.after ?? pc.dim('(deleted)')}`
|
|
84
|
+
: pc.dim(d.status);
|
|
85
|
+
console.log(` ${marker} ${d.key.padEnd(keyWidth)} ${detail}`);
|
|
86
|
+
}
|
|
87
|
+
if (!showValues) {
|
|
88
|
+
console.log(` ${pc.dim('(values hidden — pass --show-values to print them)')}`);
|
|
72
89
|
}
|
|
73
90
|
}
|
|
74
91
|
async function runSync() {
|
|
@@ -89,6 +106,8 @@ ${pc.bold('hazo-env sync')} — local Postgres/files sync (pg_dump/pg_restore/ta
|
|
|
89
106
|
hazo-env sync upload-files --archive <path> Restore an archive into files_root + .env.local
|
|
90
107
|
--confirm --allow-prod-target Required to apply (else prints a diff preview)
|
|
91
108
|
--skip-env Restore files only; leave .env.local untouched
|
|
109
|
+
--prune-env Delete .env.local keys the archive doesn't have
|
|
110
|
+
--show-values Print env values in the diff (hidden by default)
|
|
92
111
|
`);
|
|
93
112
|
return;
|
|
94
113
|
}
|
|
@@ -109,7 +128,7 @@ ${pc.bold('hazo-env sync')} — local Postgres/files sync (pg_dump/pg_restore/ta
|
|
|
109
128
|
const dumpIdx = subArgs.findIndex((a) => a === '--dump');
|
|
110
129
|
const dumpPath = dumpIdx >= 0 ? subArgs[dumpIdx + 1] : undefined;
|
|
111
130
|
const confirm = subArgs.includes('--confirm');
|
|
112
|
-
const allowProd = subArgs.includes('--allow-prod-target');
|
|
131
|
+
const allowProd = subArgs.includes('--allow-prod-target') || cfg.allowProd === true;
|
|
113
132
|
if (!dumpPath) {
|
|
114
133
|
console.error(pc.red('Error: sync upload-db requires --dump <path>'));
|
|
115
134
|
process.exit(1);
|
|
@@ -128,20 +147,22 @@ ${pc.bold('hazo-env sync')} — local Postgres/files sync (pg_dump/pg_restore/ta
|
|
|
128
147
|
const archiveIdx = subArgs.findIndex((a) => a === '--archive');
|
|
129
148
|
const archivePath = archiveIdx >= 0 ? subArgs[archiveIdx + 1] : undefined;
|
|
130
149
|
const confirm = subArgs.includes('--confirm');
|
|
131
|
-
const allowProd = subArgs.includes('--allow-prod-target');
|
|
150
|
+
const allowProd = subArgs.includes('--allow-prod-target') || cfg.allowProd === true;
|
|
132
151
|
const skipEnv = subArgs.includes('--skip-env');
|
|
152
|
+
const pruneEnv = subArgs.includes('--prune-env');
|
|
153
|
+
const showValues = subArgs.includes('--show-values');
|
|
133
154
|
if (!archivePath) {
|
|
134
155
|
console.error(pc.red('Error: sync upload-files requires --archive <path>'));
|
|
135
156
|
process.exit(1);
|
|
136
157
|
}
|
|
137
158
|
console.log(`\n${pc.bold('hazo-env sync upload-files')} ${pc.dim(archivePath)}${skipEnv ? pc.dim(' (--skip-env: .env.local left untouched)') : ''}${confirm ? '' : pc.dim(' (preview — pass --confirm to apply)')}\n`);
|
|
138
|
-
const result = await withLock(cfg.work_dir, 'upload_files', () => uploadFiles(cfg, archivePath, { confirm, allowProd, skipEnv, onProgress }));
|
|
159
|
+
const result = await withLock(cfg.work_dir, 'upload_files', () => uploadFiles(cfg, archivePath, { confirm, allowProd, skipEnv, pruneEnv, onProgress }));
|
|
139
160
|
console.log('');
|
|
140
161
|
if (skipEnv) {
|
|
141
162
|
console.log(` ${pc.dim('· .env.local skipped (--skip-env) — no env diff computed, target left as-is')}`);
|
|
142
163
|
}
|
|
143
164
|
else {
|
|
144
|
-
printEnvOverrideDiff(result.diff);
|
|
165
|
+
printEnvOverrideDiff(result.diff, showValues);
|
|
145
166
|
}
|
|
146
167
|
if (result.ok) {
|
|
147
168
|
console.log(`\n ${pc.green('✓')} Restored -> ${cfg.files_root}\n`);
|
|
@@ -197,6 +218,8 @@ Usage:
|
|
|
197
218
|
hazo-env sync upload-files --archive <p> Restore an archive into files_root + .env.local
|
|
198
219
|
--confirm --allow-prod-target Required to apply (else prints a diff preview)
|
|
199
220
|
--skip-env Restore files only; leave .env.local untouched
|
|
221
|
+
--prune-env Delete .env.local keys the archive doesn't have
|
|
222
|
+
--show-values Print env values in the diff (hidden by default)
|
|
200
223
|
hazo-env serve Start the envsync HTTP service (HAZO_ENVSYNC_TOKEN required)
|
|
201
224
|
--port <n> --bind <host> Override HAZO_ENVSYNC_PORT / HAZO_ENVSYNC_BIND
|
|
202
225
|
`);
|
|
@@ -1,7 +1,15 @@
|
|
|
1
|
+
export type EnvDiffStatus = 'added' | 'changed' | 'unchanged' | 'removed';
|
|
1
2
|
export interface EnvOverrideDiff {
|
|
2
3
|
key: string;
|
|
3
4
|
before: string | undefined;
|
|
4
|
-
|
|
5
|
+
/** undefined when the key is about to be deleted (status 'removed'). */
|
|
6
|
+
after: string | undefined;
|
|
7
|
+
status: EnvDiffStatus;
|
|
8
|
+
}
|
|
9
|
+
/** A diff with every value stripped — the only shape allowed to leave the process. */
|
|
10
|
+
export interface RedactedEnvDiff {
|
|
11
|
+
key: string;
|
|
12
|
+
status: EnvDiffStatus;
|
|
5
13
|
}
|
|
6
14
|
/** Parse dotenv-style KEY=VALUE text into a plain record. */
|
|
7
15
|
export declare function parseDotenvText(text: string): Record<string, string>;
|
|
@@ -10,6 +18,21 @@ export declare function parseDotenvText(text: string): Record<string, string>;
|
|
|
10
18
|
* Overrides always win.
|
|
11
19
|
*/
|
|
12
20
|
export declare function mergeEnvOverrides(archiveEnv: Record<string, string>, overrides: Record<string, string>): Record<string, string>;
|
|
21
|
+
/**
|
|
22
|
+
* The env that will actually be written to the target `.env.local`.
|
|
23
|
+
*
|
|
24
|
+
* Default (pruneMissing false): keys that exist ONLY in the target are
|
|
25
|
+
* PRESERVED. The archive's env is not authoritative over the destination —
|
|
26
|
+
* a target commonly carries local-only keys (a laptop's dev credentials, a
|
|
27
|
+
* staging-only feature flag) that nothing in the archive knows about, and
|
|
28
|
+
* silently dropping them on a files restore is data loss.
|
|
29
|
+
*
|
|
30
|
+
* With pruneMissing true the merged env becomes authoritative and target-only
|
|
31
|
+
* keys are deleted — an explicit opt-in (`--prune-env` / `pruneEnv: true`).
|
|
32
|
+
*/
|
|
33
|
+
export declare function resolveFinalEnv(merged: Record<string, string>, currentEnv: Record<string, string>, opts?: {
|
|
34
|
+
pruneMissing?: boolean;
|
|
35
|
+
}): Record<string, string>;
|
|
13
36
|
/**
|
|
14
37
|
* Diff the merged (about-to-be-written) env against the CURRENT target
|
|
15
38
|
* .env.local content — this is "what's about to change" from the
|
|
@@ -17,8 +40,28 @@ export declare function mergeEnvOverrides(archiveEnv: Record<string, string>, ov
|
|
|
17
40
|
* to overwrite. (Diffing against the archive's original values instead
|
|
18
41
|
* would show "what changed since the download", which is less useful when
|
|
19
42
|
* deciding whether to confirm an upload.)
|
|
43
|
+
*
|
|
44
|
+
* The diff covers the UNION of both key sets, not just the merged one: a key
|
|
45
|
+
* that exists only in the target is exactly the case an operator most needs
|
|
46
|
+
* to see before confirming, since under `pruneMissing` it is about to be
|
|
47
|
+
* deleted. Every entry carries an explicit status so a removal can never be
|
|
48
|
+
* mistaken for a no-op.
|
|
49
|
+
*/
|
|
50
|
+
export declare function diffEnvAgainstCurrent(merged: Record<string, string>, currentEnv: Record<string, string>, opts?: {
|
|
51
|
+
pruneMissing?: boolean;
|
|
52
|
+
}): EnvOverrideDiff[];
|
|
53
|
+
/**
|
|
54
|
+
* Strip every value out of a diff, keeping only key + status. Anything that
|
|
55
|
+
* leaves the process (HTTP response body, control page, log line) must go
|
|
56
|
+
* through this — `.env.local` values are secrets by definition.
|
|
57
|
+
*/
|
|
58
|
+
export declare function redactEnvDiff(diff: EnvOverrideDiff[]): RedactedEnvDiff[];
|
|
59
|
+
/**
|
|
60
|
+
* Serialize a record back into dotenv text. Round-trips through
|
|
61
|
+
* parseDotenvText for any value — spaces, `#`, `=`, quotes and newlines all
|
|
62
|
+
* survive (see quoteDotenvValue). Writing bare `KEY=value` here used to
|
|
63
|
+
* silently corrupt PEM keys, JSON blobs and anything with a comment
|
|
64
|
+
* character in it.
|
|
20
65
|
*/
|
|
21
|
-
export declare function diffEnvAgainstCurrent(merged: Record<string, string>, currentEnv: Record<string, string>): EnvOverrideDiff[];
|
|
22
|
-
/** Serialize a record back into KEY=VALUE dotenv text. */
|
|
23
66
|
export declare function serializeDotenv(env: Record<string, string>): string;
|
|
24
67
|
//# sourceMappingURL=dotenv-merge.d.ts.map
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"dotenv-merge.d.ts","sourceRoot":"","sources":["../../src/envsync/dotenv-merge.ts"],"names":[],"mappings":"AAOA,MAAM,WAAW,eAAe;IAC9B,GAAG,EAAE,MAAM,CAAC;IACZ,MAAM,EAAE,MAAM,GAAG,SAAS,CAAC;IAC3B,KAAK,EAAE,MAAM,CAAC;
|
|
1
|
+
{"version":3,"file":"dotenv-merge.d.ts","sourceRoot":"","sources":["../../src/envsync/dotenv-merge.ts"],"names":[],"mappings":"AAOA,MAAM,MAAM,aAAa,GAAG,OAAO,GAAG,SAAS,GAAG,WAAW,GAAG,SAAS,CAAC;AAE1E,MAAM,WAAW,eAAe;IAC9B,GAAG,EAAE,MAAM,CAAC;IACZ,MAAM,EAAE,MAAM,GAAG,SAAS,CAAC;IAC3B,wEAAwE;IACxE,KAAK,EAAE,MAAM,GAAG,SAAS,CAAC;IAC1B,MAAM,EAAE,aAAa,CAAC;CACvB;AAED,sFAAsF;AACtF,MAAM,WAAW,eAAe;IAC9B,GAAG,EAAE,MAAM,CAAC;IACZ,MAAM,EAAE,aAAa,CAAC;CACvB;AAED,6DAA6D;AAC7D,wBAAgB,eAAe,CAAC,IAAI,EAAE,MAAM,GAAG,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,CAEpE;AAED;;;GAGG;AACH,wBAAgB,iBAAiB,CAC/B,UAAU,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,EAClC,SAAS,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,GAChC,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,CAExB;AAED;;;;;;;;;;;GAWG;AACH,wBAAgB,eAAe,CAC7B,MAAM,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,EAC9B,UAAU,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,EAClC,IAAI,CAAC,EAAE;IAAE,YAAY,CAAC,EAAE,OAAO,CAAA;CAAE,GAChC,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,CAExB;AAED;;;;;;;;;;;;;GAaG;AACH,wBAAgB,qBAAqB,CACnC,MAAM,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,EAC9B,UAAU,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,EAClC,IAAI,CAAC,EAAE;IAAE,YAAY,CAAC,EAAE,OAAO,CAAA;CAAE,GAChC,eAAe,EAAE,CAgBnB;AAED;;;;GAIG;AACH,wBAAgB,aAAa,CAAC,IAAI,EAAE,eAAe,EAAE,GAAG,eAAe,EAAE,CAExE;AA2CD;;;;;;GAMG;AACH,wBAAgB,eAAe,CAAC,GAAG,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,GAAG,MAAM,CAInE"}
|
|
@@ -14,6 +14,21 @@ export function parseDotenvText(text) {
|
|
|
14
14
|
export function mergeEnvOverrides(archiveEnv, overrides) {
|
|
15
15
|
return { ...archiveEnv, ...overrides };
|
|
16
16
|
}
|
|
17
|
+
/**
|
|
18
|
+
* The env that will actually be written to the target `.env.local`.
|
|
19
|
+
*
|
|
20
|
+
* Default (pruneMissing false): keys that exist ONLY in the target are
|
|
21
|
+
* PRESERVED. The archive's env is not authoritative over the destination —
|
|
22
|
+
* a target commonly carries local-only keys (a laptop's dev credentials, a
|
|
23
|
+
* staging-only feature flag) that nothing in the archive knows about, and
|
|
24
|
+
* silently dropping them on a files restore is data loss.
|
|
25
|
+
*
|
|
26
|
+
* With pruneMissing true the merged env becomes authoritative and target-only
|
|
27
|
+
* keys are deleted — an explicit opt-in (`--prune-env` / `pruneEnv: true`).
|
|
28
|
+
*/
|
|
29
|
+
export function resolveFinalEnv(merged, currentEnv, opts) {
|
|
30
|
+
return opts?.pruneMissing ? { ...merged } : { ...currentEnv, ...merged };
|
|
31
|
+
}
|
|
17
32
|
/**
|
|
18
33
|
* Diff the merged (about-to-be-written) env against the CURRENT target
|
|
19
34
|
* .env.local content — this is "what's about to change" from the
|
|
@@ -21,15 +36,86 @@ export function mergeEnvOverrides(archiveEnv, overrides) {
|
|
|
21
36
|
* to overwrite. (Diffing against the archive's original values instead
|
|
22
37
|
* would show "what changed since the download", which is less useful when
|
|
23
38
|
* deciding whether to confirm an upload.)
|
|
39
|
+
*
|
|
40
|
+
* The diff covers the UNION of both key sets, not just the merged one: a key
|
|
41
|
+
* that exists only in the target is exactly the case an operator most needs
|
|
42
|
+
* to see before confirming, since under `pruneMissing` it is about to be
|
|
43
|
+
* deleted. Every entry carries an explicit status so a removal can never be
|
|
44
|
+
* mistaken for a no-op.
|
|
45
|
+
*/
|
|
46
|
+
export function diffEnvAgainstCurrent(merged, currentEnv, opts) {
|
|
47
|
+
const keys = Array.from(new Set([...Object.keys(merged), ...Object.keys(currentEnv)])).sort();
|
|
48
|
+
return keys.map((key) => {
|
|
49
|
+
const before = currentEnv[key];
|
|
50
|
+
if (!(key in merged)) {
|
|
51
|
+
// Target-only key: deleted under pruneMissing, otherwise carried over.
|
|
52
|
+
return opts?.pruneMissing
|
|
53
|
+
? { key, before, after: undefined, status: 'removed' }
|
|
54
|
+
: { key, before, after: before, status: 'unchanged' };
|
|
55
|
+
}
|
|
56
|
+
const after = merged[key];
|
|
57
|
+
const status = before === undefined ? 'added' : before === after ? 'unchanged' : 'changed';
|
|
58
|
+
return { key, before, after, status };
|
|
59
|
+
});
|
|
60
|
+
}
|
|
61
|
+
/**
|
|
62
|
+
* Strip every value out of a diff, keeping only key + status. Anything that
|
|
63
|
+
* leaves the process (HTTP response body, control page, log line) must go
|
|
64
|
+
* through this — `.env.local` values are secrets by definition.
|
|
24
65
|
*/
|
|
25
|
-
export function
|
|
26
|
-
return
|
|
27
|
-
.sort()
|
|
28
|
-
.map((key) => ({ key, before: currentEnv[key], after: merged[key] }));
|
|
66
|
+
export function redactEnvDiff(diff) {
|
|
67
|
+
return diff.map(({ key, status }) => ({ key, status }));
|
|
29
68
|
}
|
|
30
|
-
|
|
69
|
+
// Values made only of these characters need no quoting: dotenv's unquoted
|
|
70
|
+
// value grammar (`[^#\r\n]+`, then trimmed) round-trips them exactly.
|
|
71
|
+
// Deliberately conservative — anything else gets quoted rather than reasoned
|
|
72
|
+
// about case by case.
|
|
73
|
+
const BARE_SAFE = /^[A-Za-z0-9_@%^,.:/+-]*$/;
|
|
74
|
+
/**
|
|
75
|
+
* Quote a single value so that `dotenv.parse` returns it byte-for-byte.
|
|
76
|
+
*
|
|
77
|
+
* Strategy, in order of preference:
|
|
78
|
+
* 1. bare — only for the conservative safe charset above.
|
|
79
|
+
* 2. single quotes — dotenv performs NO escape processing inside `'...'`,
|
|
80
|
+
* so the content is fully literal (spaces, `#`, `=`, `"`, backslashes,
|
|
81
|
+
* real newlines: a PEM key or a JSON blob lands here). Requires no `'`.
|
|
82
|
+
* 3. backticks — same literal semantics as single quotes. Requires no backtick.
|
|
83
|
+
* 4. double quotes — the only form dotenv post-processes (`\n`/`\r` escapes
|
|
84
|
+
* become real newlines), so it is used last, and only when the value has
|
|
85
|
+
* no `"` and no literal `\n`/`\r` two-char sequence that the un-escaping
|
|
86
|
+
* would corrupt.
|
|
87
|
+
*
|
|
88
|
+
* A literal carriage return can only survive as a `\r` escape (dotenv
|
|
89
|
+
* normalises every real CR in the source to LF before parsing), so a CR
|
|
90
|
+
* forces the double-quoted form regardless of the other quote characters.
|
|
91
|
+
*
|
|
92
|
+
* A value that needs a form we cannot produce (e.g. it mixes all three quote
|
|
93
|
+
* characters) cannot be represented losslessly in dotenv's grammar; we throw
|
|
94
|
+
* rather than silently corrupt it.
|
|
95
|
+
*/
|
|
96
|
+
function quoteDotenvValue(key, value) {
|
|
97
|
+
if (BARE_SAFE.test(value))
|
|
98
|
+
return value;
|
|
99
|
+
const hasCarriageReturn = value.includes('\r');
|
|
100
|
+
if (!hasCarriageReturn && !value.includes("'"))
|
|
101
|
+
return `'${value}'`;
|
|
102
|
+
if (!hasCarriageReturn && !value.includes('`'))
|
|
103
|
+
return `\`${value}\``;
|
|
104
|
+
if (!value.includes('"') && !/\\[nr]/.test(value)) {
|
|
105
|
+
return `"${value.replace(/\r/g, '\\r').replace(/\n/g, '\\n')}"`;
|
|
106
|
+
}
|
|
107
|
+
throw new Error(`Refusing: env value for "${key}" cannot be encoded losslessly in .env.local format (it mixes quote characters and/or escape sequences dotenv cannot round-trip).`);
|
|
108
|
+
}
|
|
109
|
+
/**
|
|
110
|
+
* Serialize a record back into dotenv text. Round-trips through
|
|
111
|
+
* parseDotenvText for any value — spaces, `#`, `=`, quotes and newlines all
|
|
112
|
+
* survive (see quoteDotenvValue). Writing bare `KEY=value` here used to
|
|
113
|
+
* silently corrupt PEM keys, JSON blobs and anything with a comment
|
|
114
|
+
* character in it.
|
|
115
|
+
*/
|
|
31
116
|
export function serializeDotenv(env) {
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
117
|
+
const entries = Object.entries(env);
|
|
118
|
+
if (!entries.length)
|
|
119
|
+
return '';
|
|
120
|
+
return entries.map(([key, value]) => `${key}=${quoteDotenvValue(key, value)}`).join('\n') + '\n';
|
|
35
121
|
}
|
package/dist/envsync/engine.d.ts
CHANGED
|
@@ -18,6 +18,12 @@ export declare function downloadDb(cfg: EnvsyncConfig, opts?: {
|
|
|
18
18
|
* Drop, recreate, and pg_restore the configured target_db from dumpPath.
|
|
19
19
|
* Guarded by assertConfirmed + assertNotProd — never runs unconfirmed or
|
|
20
20
|
* against a target that looks like production.
|
|
21
|
+
*
|
|
22
|
+
* Ordering is a safety property, not a style choice: guards, then FULL
|
|
23
|
+
* validation of dumpPath (exists, non-empty, regular file, inside work_dir,
|
|
24
|
+
* not option-shaped), and only then anything destructive. A typo'd --dump
|
|
25
|
+
* used to drop the target database before pg_restore discovered the file
|
|
26
|
+
* wasn't there.
|
|
21
27
|
*/
|
|
22
28
|
export declare function uploadDb(cfg: EnvsyncConfig, dumpPath: string, opts: {
|
|
23
29
|
confirm?: boolean;
|
|
@@ -62,6 +68,11 @@ export declare function downloadFiles(cfg: EnvsyncConfig, opts?: {
|
|
|
62
68
|
* so it's the meaningful "is this prod" check available here), then backs
|
|
63
69
|
* up the existing target .env.local, wipes and repopulates files_root from
|
|
64
70
|
* the archive, and writes the merged .env.local.
|
|
71
|
+
*
|
|
72
|
+
* The archive is fully validated (path inside work_dir, non-empty, and `tar
|
|
73
|
+
* -t` lists it cleanly) BEFORE files_root is wiped — a bad --archive used to
|
|
74
|
+
* delete the destination first and fail afterwards, which under --skip-env
|
|
75
|
+
* (nothing else reads the archive) meant total data loss.
|
|
65
76
|
*/
|
|
66
77
|
export declare function uploadFiles(cfg: EnvsyncConfig, archivePath: string, opts: {
|
|
67
78
|
confirm?: boolean;
|
|
@@ -74,6 +85,13 @@ export declare function uploadFiles(cfg: EnvsyncConfig, archivePath: string, opt
|
|
|
74
85
|
* empty because no env comparison is performed.
|
|
75
86
|
*/
|
|
76
87
|
skipEnv?: boolean;
|
|
88
|
+
/**
|
|
89
|
+
* Delete keys that exist only in the target `.env.local` (i.e. make the
|
|
90
|
+
* archive+overrides env authoritative). Off by default: target-only keys
|
|
91
|
+
* are preserved, because dropping a destination's local-only secrets is
|
|
92
|
+
* data loss that the old diff didn't even surface. Ignored under skipEnv.
|
|
93
|
+
*/
|
|
94
|
+
pruneEnv?: boolean;
|
|
77
95
|
onProgress?: (msg: string) => void;
|
|
78
96
|
}): Promise<{
|
|
79
97
|
ok: true;
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"engine.d.ts","sourceRoot":"","sources":["../../src/envsync/engine.ts"],"names":[],"mappings":"AAaA,OAAO,
|
|
1
|
+
{"version":3,"file":"engine.d.ts","sourceRoot":"","sources":["../../src/envsync/engine.ts"],"names":[],"mappings":"AAaA,OAAO,EAML,KAAK,eAAe,EACrB,MAAM,mBAAmB,CAAC;AAG3B,OAAO,KAAK,EAAE,aAAa,EAAE,MAAM,uBAAuB,CAAC;AAE3D,YAAY,EAAE,eAAe,EAAE,CAAC;AAEhC,MAAM,WAAW,iBAAiB;IAChC,EAAE,EAAE,MAAM,CAAC;IACX,IAAI,EAAE,MAAM,CAAC;IACb,KAAK,EAAE,MAAM,CAAC;CACf;AAaD;;GAEG;AACH,wBAAsB,UAAU,CAC9B,GAAG,EAAE,aAAa,EAClB,IAAI,CAAC,EAAE;IAAE,UAAU,CAAC,EAAE,CAAC,GAAG,EAAE,MAAM,KAAK,IAAI,CAAC;IAAC,EAAE,CAAC,EAAE,MAAM,CAAC;IAAC,GAAG,CAAC,EAAE,MAAM,MAAM,CAAA;CAAE,GAC7E,OAAO,CAAC,iBAAiB,CAAC,CAe5B;AAED;;;;;;;;;;GAUG;AACH,wBAAsB,QAAQ,CAC5B,GAAG,EAAE,aAAa,EAClB,QAAQ,EAAE,MAAM,EAChB,IAAI,EAAE;IAAE,OAAO,CAAC,EAAE,OAAO,CAAC;IAAC,SAAS,CAAC,EAAE,OAAO,CAAC;IAAC,UAAU,CAAC,EAAE,CAAC,GAAG,EAAE,MAAM,KAAK,IAAI,CAAA;CAAE,GACnF,OAAO,CAAC;IAAE,EAAE,EAAE,IAAI,CAAA;CAAE,CAAC,CA+BvB;AAED;;;;;;;;;;;GAWG;AACH,wBAAsB,aAAa,CACjC,GAAG,EAAE,aAAa,EAClB,IAAI,CAAC,EAAE;IAAE,UAAU,CAAC,EAAE,CAAC,GAAG,EAAE,MAAM,KAAK,IAAI,CAAC;IAAC,EAAE,CAAC,EAAE,MAAM,CAAC;IAAC,GAAG,CAAC,EAAE,MAAM,MAAM,CAAA;CAAE,GAC7E,OAAO,CAAC,iBAAiB,CAAC,CA2B5B;AAED;;;;;;;;;;;;;;;;;;;;;;;;GAwBG;AACH,wBAAsB,WAAW,CAC/B,GAAG,EAAE,aAAa,EAClB,WAAW,EAAE,MAAM,EACnB,IAAI,EAAE;IACJ,OAAO,CAAC,EAAE,OAAO,CAAC;IAClB,SAAS,CAAC,EAAE,OAAO,CAAC;IACpB;;;;;;OAMG;IACH,OAAO,CAAC,EAAE,OAAO,CAAC;IAClB;;;;;OAKG;IACH,QAAQ,CAAC,EAAE,OAAO,CAAC;IACnB,UAAU,CAAC,EAAE,CAAC,GAAG,EAAE,MAAM,KAAK,IAAI,CAAC;CACpC,GACA,OAAO,CAAC;IAAE,EAAE,EAAE,IAAI,CAAC;IAAC,IAAI,EAAE,eAAe,EAAE,CAAA;CAAE,GAAG;IAAE,EAAE,EAAE,KAAK,CAAC;IAAC,IAAI,EAAE,eAAe,EAAE,CAAA;CAAE,CAAC,CAiFzF"}
|
package/dist/envsync/engine.js
CHANGED
|
@@ -10,7 +10,9 @@ import path from 'node:path';
|
|
|
10
10
|
import { runCmd, runShell } from './exec.js';
|
|
11
11
|
import { assertFreeSpace } from './retention.js';
|
|
12
12
|
import { assertConfirmed, assertNotProd } from './guards.js';
|
|
13
|
-
import { mergeEnvOverrides, diffEnvAgainstCurrent, parseDotenvText, serializeDotenv, } from './dotenv-merge.js';
|
|
13
|
+
import { mergeEnvOverrides, diffEnvAgainstCurrent, parseDotenvText, resolveFinalEnv, serializeDotenv, } from './dotenv-merge.js';
|
|
14
|
+
import { resolveArtifactPath } from './paths.js';
|
|
15
|
+
import { redactingProgress, redactConnectionString } from './redact.js';
|
|
14
16
|
// Free-space floor checked before writing a new dump/archive into work_dir.
|
|
15
17
|
// Not configurable in Phase 1 — a fixed conservative floor is good enough
|
|
16
18
|
// to catch "disk is basically full" before pg_dump/tar fail mid-write.
|
|
@@ -27,8 +29,11 @@ export async function downloadDb(cfg, opts) {
|
|
|
27
29
|
fs.mkdirSync(cfg.work_dir, { recursive: true });
|
|
28
30
|
const id = opts?.id ?? defaultId(opts?.now);
|
|
29
31
|
const filePath = path.join(cfg.work_dir, `db-${id}.pgdump`);
|
|
30
|
-
|
|
31
|
-
|
|
32
|
+
// source_db is routinely a full postgres://user:pass@host URI — never let
|
|
33
|
+
// it (or pg_dump's own echo of it on stderr) reach a progress sink raw.
|
|
34
|
+
const onProgress = redactingProgress(opts?.onProgress);
|
|
35
|
+
onProgress?.(`Dumping ${redactConnectionString(cfg.source_db)} -> ${filePath}`);
|
|
36
|
+
await runCmd('pg_dump', ['-Fc', cfg.source_db, '-f', filePath], { onProgress });
|
|
32
37
|
const bytes = fs.statSync(filePath).size;
|
|
33
38
|
return { id, path: filePath, bytes };
|
|
34
39
|
}
|
|
@@ -36,23 +41,34 @@ export async function downloadDb(cfg, opts) {
|
|
|
36
41
|
* Drop, recreate, and pg_restore the configured target_db from dumpPath.
|
|
37
42
|
* Guarded by assertConfirmed + assertNotProd — never runs unconfirmed or
|
|
38
43
|
* against a target that looks like production.
|
|
44
|
+
*
|
|
45
|
+
* Ordering is a safety property, not a style choice: guards, then FULL
|
|
46
|
+
* validation of dumpPath (exists, non-empty, regular file, inside work_dir,
|
|
47
|
+
* not option-shaped), and only then anything destructive. A typo'd --dump
|
|
48
|
+
* used to drop the target database before pg_restore discovered the file
|
|
49
|
+
* wasn't there.
|
|
39
50
|
*/
|
|
40
51
|
export async function uploadDb(cfg, dumpPath, opts) {
|
|
41
52
|
assertConfirmed(opts.confirm, cfg.target_db);
|
|
42
|
-
assertNotProd(cfg.target_db, cfg.prodDbNames, opts.allowProd);
|
|
53
|
+
assertNotProd(cfg.target_db, cfg.prodDbNames, opts.allowProd, cfg.prodDbPattern);
|
|
54
|
+
const safeDumpPath = resolveArtifactPath(dumpPath, cfg.work_dir, 'dump path');
|
|
55
|
+
const onProgress = redactingProgress(opts.onProgress);
|
|
56
|
+
const safeTargetName = redactConnectionString(cfg.target_db);
|
|
43
57
|
if (cfg.pre_restore_cmd) {
|
|
44
|
-
|
|
45
|
-
await runShell(cfg.pre_restore_cmd, { onProgress
|
|
58
|
+
onProgress?.('Running pre_restore_cmd...');
|
|
59
|
+
await runShell(cfg.pre_restore_cmd, { onProgress });
|
|
46
60
|
}
|
|
47
|
-
|
|
48
|
-
await runCmd('dropdb', ['--if-exists', cfg.target_db], { onProgress
|
|
49
|
-
|
|
50
|
-
await runCmd('createdb', ['-O', cfg.owner, cfg.target_db], { onProgress
|
|
51
|
-
|
|
52
|
-
|
|
61
|
+
onProgress?.(`Dropping ${safeTargetName}...`);
|
|
62
|
+
await runCmd('dropdb', ['--if-exists', cfg.target_db], { onProgress });
|
|
63
|
+
onProgress?.(`Creating ${safeTargetName}...`);
|
|
64
|
+
await runCmd('createdb', ['-O', cfg.owner, cfg.target_db], { onProgress });
|
|
65
|
+
onProgress?.(`Restoring ${safeDumpPath} -> ${safeTargetName}...`);
|
|
66
|
+
// `--` ends option parsing: without it a dump path is a bare positional
|
|
67
|
+
// operand and pg_restore would happily read "--jobs=8" as a flag.
|
|
68
|
+
await runCmd('pg_restore', ['-d', cfg.target_db, '--', safeDumpPath], { onProgress });
|
|
53
69
|
if (cfg.post_restore_cmd) {
|
|
54
|
-
|
|
55
|
-
await runShell(cfg.post_restore_cmd, { onProgress
|
|
70
|
+
onProgress?.('Running post_restore_cmd...');
|
|
71
|
+
await runShell(cfg.post_restore_cmd, { onProgress });
|
|
56
72
|
}
|
|
57
73
|
return { ok: true };
|
|
58
74
|
}
|
|
@@ -74,13 +90,14 @@ export async function downloadFiles(cfg, opts) {
|
|
|
74
90
|
fs.mkdirSync(cfg.files_root, { recursive: true });
|
|
75
91
|
const id = opts?.id ?? defaultId(opts?.now);
|
|
76
92
|
const filePath = path.join(cfg.work_dir, `files-${id}.tar.gz`);
|
|
93
|
+
const onProgress = redactingProgress(opts?.onProgress);
|
|
77
94
|
const stageDir = fs.mkdtempSync(path.join(os.tmpdir(), 'hazo_envsync_stage_'));
|
|
78
95
|
try {
|
|
79
96
|
const envLocalSrc = path.join(process.cwd(), '.env.local');
|
|
80
97
|
const stagedEnvLocal = path.join(stageDir, ENV_LOCAL_MEMBER);
|
|
81
98
|
fs.writeFileSync(stagedEnvLocal, fs.existsSync(envLocalSrc) ? fs.readFileSync(envLocalSrc) : '');
|
|
82
|
-
|
|
83
|
-
await runCmd('tar', ['-czf', filePath, '-C', cfg.files_root, '.', '-C', stageDir, ENV_LOCAL_MEMBER], { onProgress
|
|
99
|
+
onProgress?.(`Archiving ${cfg.files_root} -> ${filePath}`);
|
|
100
|
+
await runCmd('tar', ['-czf', filePath, '-C', cfg.files_root, '.', '-C', stageDir, ENV_LOCAL_MEMBER], { onProgress });
|
|
84
101
|
}
|
|
85
102
|
finally {
|
|
86
103
|
fs.rmSync(stageDir, { recursive: true, force: true });
|
|
@@ -107,20 +124,36 @@ export async function downloadFiles(cfg, opts) {
|
|
|
107
124
|
* so it's the meaningful "is this prod" check available here), then backs
|
|
108
125
|
* up the existing target .env.local, wipes and repopulates files_root from
|
|
109
126
|
* the archive, and writes the merged .env.local.
|
|
127
|
+
*
|
|
128
|
+
* The archive is fully validated (path inside work_dir, non-empty, and `tar
|
|
129
|
+
* -t` lists it cleanly) BEFORE files_root is wiped — a bad --archive used to
|
|
130
|
+
* delete the destination first and fail afterwards, which under --skip-env
|
|
131
|
+
* (nothing else reads the archive) meant total data loss.
|
|
110
132
|
*/
|
|
111
133
|
export async function uploadFiles(cfg, archivePath, opts) {
|
|
112
134
|
const targetEnvLocalPath = path.join(process.cwd(), '.env.local');
|
|
135
|
+
const onProgress = redactingProgress(opts.onProgress);
|
|
136
|
+
// ── validate the source artifact before ANY destructive step ────────────
|
|
137
|
+
const safeArchivePath = resolveArtifactPath(archivePath, cfg.work_dir, 'archive path');
|
|
138
|
+
try {
|
|
139
|
+
// List-only pass: proves the file is a readable tar.gz. No onProgress —
|
|
140
|
+
// this would otherwise dump every member name into the operator's log.
|
|
141
|
+
await runCmd('tar', ['-tzf', safeArchivePath]);
|
|
142
|
+
}
|
|
143
|
+
catch {
|
|
144
|
+
throw new Error(`Refusing: archive path "${archivePath}" is not a readable tar.gz archive — nothing was modified.`);
|
|
145
|
+
}
|
|
113
146
|
// When skipEnv is set we never read/merge/diff the archive's env member —
|
|
114
147
|
// the env half of upload-files is opted out of entirely, so the diff is
|
|
115
148
|
// empty and no .env.local is ever backed up or written below.
|
|
116
149
|
let diff = [];
|
|
117
|
-
let
|
|
150
|
+
let mergedEnvText = '';
|
|
118
151
|
if (!opts.skipEnv) {
|
|
119
152
|
const extractDir = fs.mkdtempSync(path.join(os.tmpdir(), 'hazo_envsync_extract_'));
|
|
120
153
|
let archiveEnvText = '';
|
|
121
154
|
try {
|
|
122
|
-
await runCmd('tar', ['-xzf',
|
|
123
|
-
onProgress
|
|
155
|
+
await runCmd('tar', ['-xzf', safeArchivePath, '-C', extractDir, ENV_LOCAL_MEMBER], {
|
|
156
|
+
onProgress,
|
|
124
157
|
});
|
|
125
158
|
const extractedEnvLocal = path.join(extractDir, ENV_LOCAL_MEMBER);
|
|
126
159
|
if (fs.existsSync(extractedEnvLocal)) {
|
|
@@ -131,26 +164,30 @@ export async function uploadFiles(cfg, archivePath, opts) {
|
|
|
131
164
|
fs.rmSync(extractDir, { recursive: true, force: true });
|
|
132
165
|
}
|
|
133
166
|
const archiveEnv = parseDotenvText(archiveEnvText);
|
|
134
|
-
merged = mergeEnvOverrides(archiveEnv, cfg.envOverrides);
|
|
167
|
+
const merged = mergeEnvOverrides(archiveEnv, cfg.envOverrides);
|
|
135
168
|
const currentEnvText = fs.existsSync(targetEnvLocalPath)
|
|
136
169
|
? fs.readFileSync(targetEnvLocalPath, 'utf8')
|
|
137
170
|
: '';
|
|
138
171
|
const currentEnv = parseDotenvText(currentEnvText);
|
|
139
|
-
|
|
172
|
+
const pruneMissing = opts.pruneEnv === true;
|
|
173
|
+
diff = diffEnvAgainstCurrent(merged, currentEnv, { pruneMissing });
|
|
174
|
+
// Serialize up front: an un-encodable value must fail here (dry run
|
|
175
|
+
// included), never after files_root has already been wiped.
|
|
176
|
+
mergedEnvText = serializeDotenv(resolveFinalEnv(merged, currentEnv, { pruneMissing }));
|
|
140
177
|
}
|
|
141
178
|
if (!opts.confirm) {
|
|
142
179
|
return { ok: false, diff };
|
|
143
180
|
}
|
|
144
|
-
assertNotProd(cfg.target_db, cfg.prodDbNames, opts.allowProd);
|
|
181
|
+
assertNotProd(cfg.target_db, cfg.prodDbNames, opts.allowProd, cfg.prodDbPattern);
|
|
145
182
|
if (!opts.skipEnv && fs.existsSync(targetEnvLocalPath)) {
|
|
146
183
|
const ts = Date.now();
|
|
147
184
|
fs.copyFileSync(targetEnvLocalPath, `${targetEnvLocalPath}.bak-${ts}`);
|
|
148
185
|
}
|
|
149
|
-
|
|
186
|
+
onProgress?.(`Clearing ${cfg.files_root}...`);
|
|
150
187
|
fs.rmSync(cfg.files_root, { recursive: true, force: true });
|
|
151
188
|
fs.mkdirSync(cfg.files_root, { recursive: true });
|
|
152
|
-
|
|
153
|
-
await runCmd('tar', ['-xzf',
|
|
189
|
+
onProgress?.(`Extracting ${safeArchivePath} -> ${cfg.files_root}...`);
|
|
190
|
+
await runCmd('tar', ['-xzf', safeArchivePath, '-C', cfg.files_root], { onProgress });
|
|
154
191
|
// __env.local extracts as a real file inside files_root — it isn't a real
|
|
155
192
|
// asset, so remove it rather than trying to get tar to exclude it during
|
|
156
193
|
// extraction (simplest correct approach per spec). Done regardless of
|
|
@@ -159,7 +196,7 @@ export async function uploadFiles(cfg, archivePath, opts) {
|
|
|
159
196
|
if (fs.existsSync(extractedEnvMember))
|
|
160
197
|
fs.rmSync(extractedEnvMember, { force: true });
|
|
161
198
|
if (!opts.skipEnv) {
|
|
162
|
-
fs.writeFileSync(targetEnvLocalPath,
|
|
199
|
+
fs.writeFileSync(targetEnvLocalPath, mergedEnvText);
|
|
163
200
|
}
|
|
164
201
|
return { ok: true, diff };
|
|
165
202
|
}
|
package/dist/envsync/guards.d.ts
CHANGED
|
@@ -5,8 +5,28 @@
|
|
|
5
5
|
*/
|
|
6
6
|
export declare function assertConfirmed(confirm: boolean | undefined, targetName: string): void;
|
|
7
7
|
/**
|
|
8
|
-
*
|
|
8
|
+
* Name pattern that marks a target as production.
|
|
9
|
+
*
|
|
10
|
+
* An exact-name list alone is not a guard: the only name on it used to be
|
|
11
|
+
* `source_db`, so a staging→prod push (source `myapp_staging`, target
|
|
12
|
+
* `myapp_prod`) matched nothing and sailed through. The pattern catches the
|
|
13
|
+
* conventional production suffixes/prefixes regardless of what this config's
|
|
14
|
+
* source happens to be. `(prod|production|live)` must sit on a token
|
|
15
|
+
* boundary, so `product_catalog_dev` does NOT match.
|
|
16
|
+
*/
|
|
17
|
+
export declare const DEFAULT_PROD_NAME_PATTERN = "(^|[^a-z0-9])(prod|production|live)([^a-z0-9]|$)";
|
|
18
|
+
/**
|
|
19
|
+
* True when targetName is on the configured prod-name list OR matches the
|
|
20
|
+
* production name pattern (default: DEFAULT_PROD_NAME_PATTERN, overridable
|
|
21
|
+
* via `prod_db_pattern` in [envsync]).
|
|
22
|
+
*/
|
|
23
|
+
export declare function looksLikeProd(targetName: string, prodNames: string[], pattern?: string): boolean;
|
|
24
|
+
/**
|
|
25
|
+
* Throw if targetName looks like a production target (see looksLikeProd) and
|
|
9
26
|
* the caller hasn't explicitly opted in via allowProd.
|
|
27
|
+
*
|
|
28
|
+
* `allowProd` must come from server-side config (`allow_prod` in [envsync])
|
|
29
|
+
* or a local operator's CLI flag — never straight from an HTTP request body.
|
|
10
30
|
*/
|
|
11
|
-
export declare function assertNotProd(targetName: string, prodNames: string[], allowProd: boolean | undefined): void;
|
|
31
|
+
export declare function assertNotProd(targetName: string, prodNames: string[], allowProd: boolean | undefined, pattern?: string): void;
|
|
12
32
|
//# sourceMappingURL=guards.d.ts.map
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"guards.d.ts","sourceRoot":"","sources":["../../src/envsync/guards.ts"],"names":[],"mappings":"AAEA;;;;GAIG;AACH,wBAAgB,eAAe,CAAC,OAAO,EAAE,OAAO,GAAG,SAAS,EAAE,UAAU,EAAE,MAAM,GAAG,IAAI,CAMtF;AAED
|
|
1
|
+
{"version":3,"file":"guards.d.ts","sourceRoot":"","sources":["../../src/envsync/guards.ts"],"names":[],"mappings":"AAEA;;;;GAIG;AACH,wBAAgB,eAAe,CAAC,OAAO,EAAE,OAAO,GAAG,SAAS,EAAE,UAAU,EAAE,MAAM,GAAG,IAAI,CAMtF;AAED;;;;;;;;;GASG;AACH,eAAO,MAAM,yBAAyB,qDAAqD,CAAC;AAa5F;;;;GAIG;AACH,wBAAgB,aAAa,CAAC,UAAU,EAAE,MAAM,EAAE,SAAS,EAAE,MAAM,EAAE,EAAE,OAAO,CAAC,EAAE,MAAM,GAAG,OAAO,CAIhG;AAED;;;;;;GAMG;AACH,wBAAgB,aAAa,CAC3B,UAAU,EAAE,MAAM,EAClB,SAAS,EAAE,MAAM,EAAE,EACnB,SAAS,EAAE,OAAO,GAAG,SAAS,EAC9B,OAAO,CAAC,EAAE,MAAM,GACf,IAAI,CAMN"}
|
package/dist/envsync/guards.js
CHANGED
|
@@ -10,11 +10,48 @@ export function assertConfirmed(confirm, targetName) {
|
|
|
10
10
|
}
|
|
11
11
|
}
|
|
12
12
|
/**
|
|
13
|
-
*
|
|
13
|
+
* Name pattern that marks a target as production.
|
|
14
|
+
*
|
|
15
|
+
* An exact-name list alone is not a guard: the only name on it used to be
|
|
16
|
+
* `source_db`, so a staging→prod push (source `myapp_staging`, target
|
|
17
|
+
* `myapp_prod`) matched nothing and sailed through. The pattern catches the
|
|
18
|
+
* conventional production suffixes/prefixes regardless of what this config's
|
|
19
|
+
* source happens to be. `(prod|production|live)` must sit on a token
|
|
20
|
+
* boundary, so `product_catalog_dev` does NOT match.
|
|
21
|
+
*/
|
|
22
|
+
export const DEFAULT_PROD_NAME_PATTERN = '(^|[^a-z0-9])(prod|production|live)([^a-z0-9]|$)';
|
|
23
|
+
function prodRegex(pattern) {
|
|
24
|
+
// An explicitly-empty configured pattern means "exact-name list only".
|
|
25
|
+
if (pattern !== undefined && pattern.trim() === '')
|
|
26
|
+
return null;
|
|
27
|
+
try {
|
|
28
|
+
return new RegExp(pattern ?? DEFAULT_PROD_NAME_PATTERN, 'i');
|
|
29
|
+
}
|
|
30
|
+
catch {
|
|
31
|
+
// A malformed operator-supplied pattern must never widen the guard.
|
|
32
|
+
return new RegExp(DEFAULT_PROD_NAME_PATTERN, 'i');
|
|
33
|
+
}
|
|
34
|
+
}
|
|
35
|
+
/**
|
|
36
|
+
* True when targetName is on the configured prod-name list OR matches the
|
|
37
|
+
* production name pattern (default: DEFAULT_PROD_NAME_PATTERN, overridable
|
|
38
|
+
* via `prod_db_pattern` in [envsync]).
|
|
39
|
+
*/
|
|
40
|
+
export function looksLikeProd(targetName, prodNames, pattern) {
|
|
41
|
+
if (prodNames.includes(targetName))
|
|
42
|
+
return true;
|
|
43
|
+
const re = prodRegex(pattern);
|
|
44
|
+
return re ? re.test(targetName) : false;
|
|
45
|
+
}
|
|
46
|
+
/**
|
|
47
|
+
* Throw if targetName looks like a production target (see looksLikeProd) and
|
|
14
48
|
* the caller hasn't explicitly opted in via allowProd.
|
|
49
|
+
*
|
|
50
|
+
* `allowProd` must come from server-side config (`allow_prod` in [envsync])
|
|
51
|
+
* or a local operator's CLI flag — never straight from an HTTP request body.
|
|
15
52
|
*/
|
|
16
|
-
export function assertNotProd(targetName, prodNames, allowProd) {
|
|
17
|
-
if (
|
|
18
|
-
throw new Error(`Refusing: "${targetName}" looks like a production target. Pass allowProd:true to override.`);
|
|
53
|
+
export function assertNotProd(targetName, prodNames, allowProd, pattern) {
|
|
54
|
+
if (looksLikeProd(targetName, prodNames, pattern) && !allowProd) {
|
|
55
|
+
throw new Error(`Refusing: "${targetName}" looks like a production target. Pass allowProd:true (set allow_prod = true in [envsync], or use --allow-prod-target) to override.`);
|
|
19
56
|
}
|
|
20
57
|
}
|
|
@@ -0,0 +1,10 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Validate a caller-supplied dump/archive path and return its real (symlink-
|
|
3
|
+
* resolved) absolute path.
|
|
4
|
+
*
|
|
5
|
+
* Throws (message always starts with "Refusing", so the HTTP layer maps it to
|
|
6
|
+
* 400) when the path starts with "-", does not exist, is not a regular file,
|
|
7
|
+
* is empty, or resolves outside `workDir`.
|
|
8
|
+
*/
|
|
9
|
+
export declare function resolveArtifactPath(candidate: string, workDir: string, label: string): string;
|
|
10
|
+
//# sourceMappingURL=paths.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"paths.d.ts","sourceRoot":"","sources":["../../src/envsync/paths.ts"],"names":[],"mappings":"AAiCA;;;;;;;GAOG;AACH,wBAAgB,mBAAmB,CAAC,SAAS,EAAE,MAAM,EAAE,OAAO,EAAE,MAAM,EAAE,KAAK,EAAE,MAAM,GAAG,MAAM,CAkC7F"}
|
|
@@ -0,0 +1,67 @@
|
|
|
1
|
+
// hazo_env/src/envsync/paths.ts — artifact-path validation for envsync uploads
|
|
2
|
+
//
|
|
3
|
+
// uploadDb/uploadFiles take a caller-supplied path (CLI --dump/--archive, or
|
|
4
|
+
// an HTTP request body) and hand it to pg_restore/tar. Three separate
|
|
5
|
+
// problems, all fixed here in one place, all BEFORE anything destructive runs:
|
|
6
|
+
//
|
|
7
|
+
// 1. Argument injection — a path like "--jobs=8" or "-f/etc/passwd" is a
|
|
8
|
+
// positional operand to us but an OPTION to pg_restore/tar. Callers also
|
|
9
|
+
// pass `--` as an end-of-options separator (see engine.ts), but a
|
|
10
|
+
// leading-dash path is rejected outright here as defence in depth.
|
|
11
|
+
// 2. Arbitrary read — nothing constrained the path to files envsync created.
|
|
12
|
+
// 3. Destructive-before-validate — a typo'd path used to be discovered only
|
|
13
|
+
// after dropdb/rm -rf had already run.
|
|
14
|
+
//
|
|
15
|
+
// Confinement is checked on the REAL path (symlinks resolved) so a symlink
|
|
16
|
+
// inside work_dir pointing at /etc, or a `../..` traversal, both fail. The
|
|
17
|
+
// resolved path is what callers should pass to the child process — that also
|
|
18
|
+
// closes the symlink-swap window between check and use.
|
|
19
|
+
import fs from 'node:fs';
|
|
20
|
+
import path from 'node:path';
|
|
21
|
+
function realDirOrResolve(dir) {
|
|
22
|
+
try {
|
|
23
|
+
return fs.realpathSync(dir);
|
|
24
|
+
}
|
|
25
|
+
catch {
|
|
26
|
+
// work_dir may not exist yet (nothing has been downloaded into it) — a
|
|
27
|
+
// lexical resolve is enough, since the artifact realpath check below will
|
|
28
|
+
// then never be inside it and the caller gets a clear refusal anyway.
|
|
29
|
+
return path.resolve(dir);
|
|
30
|
+
}
|
|
31
|
+
}
|
|
32
|
+
/**
|
|
33
|
+
* Validate a caller-supplied dump/archive path and return its real (symlink-
|
|
34
|
+
* resolved) absolute path.
|
|
35
|
+
*
|
|
36
|
+
* Throws (message always starts with "Refusing", so the HTTP layer maps it to
|
|
37
|
+
* 400) when the path starts with "-", does not exist, is not a regular file,
|
|
38
|
+
* is empty, or resolves outside `workDir`.
|
|
39
|
+
*/
|
|
40
|
+
export function resolveArtifactPath(candidate, workDir, label) {
|
|
41
|
+
if (candidate.startsWith('-')) {
|
|
42
|
+
throw new Error(`Refusing: ${label} "${candidate}" starts with "-" — it would be read as a command-line option, not a file.`);
|
|
43
|
+
}
|
|
44
|
+
if (candidate.includes('\0')) {
|
|
45
|
+
throw new Error(`Refusing: ${label} contains a NUL byte.`);
|
|
46
|
+
}
|
|
47
|
+
let real;
|
|
48
|
+
try {
|
|
49
|
+
real = fs.realpathSync(candidate);
|
|
50
|
+
}
|
|
51
|
+
catch {
|
|
52
|
+
throw new Error(`Refusing: ${label} "${candidate}" does not exist or is not readable.`);
|
|
53
|
+
}
|
|
54
|
+
const realWorkDir = realDirOrResolve(workDir);
|
|
55
|
+
const rel = path.relative(realWorkDir, real);
|
|
56
|
+
if (!rel || rel.startsWith('..') || path.isAbsolute(rel)) {
|
|
57
|
+
throw new Error(`Refusing: ${label} "${candidate}" resolves outside work_dir "${workDir}" — envsync only reads dumps/archives from its own work_dir.`);
|
|
58
|
+
}
|
|
59
|
+
const stat = fs.statSync(real);
|
|
60
|
+
if (!stat.isFile()) {
|
|
61
|
+
throw new Error(`Refusing: ${label} "${candidate}" is not a regular file.`);
|
|
62
|
+
}
|
|
63
|
+
if (stat.size === 0) {
|
|
64
|
+
throw new Error(`Refusing: ${label} "${candidate}" is empty.`);
|
|
65
|
+
}
|
|
66
|
+
return real;
|
|
67
|
+
}
|
|
@@ -0,0 +1,14 @@
|
|
|
1
|
+
export declare const REDACTED = "***";
|
|
2
|
+
/**
|
|
3
|
+
* Strip credentials out of a connection string (or any text that may embed
|
|
4
|
+
* one — pg_dump/pg_restore echo the URI back in their own error output, so
|
|
5
|
+
* this is applied to whole progress lines, not just to config values).
|
|
6
|
+
*/
|
|
7
|
+
export declare function redactConnectionString(value: string): string;
|
|
8
|
+
/**
|
|
9
|
+
* Wrap a progress callback so every line it receives is credential-redacted.
|
|
10
|
+
* Returns undefined when there is no callback, so it can be passed straight
|
|
11
|
+
* through to runCmd's optional `onProgress`.
|
|
12
|
+
*/
|
|
13
|
+
export declare function redactingProgress(onProgress?: (msg: string) => void): ((msg: string) => void) | undefined;
|
|
14
|
+
//# sourceMappingURL=redact.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"redact.d.ts","sourceRoot":"","sources":["../../src/envsync/redact.ts"],"names":[],"mappings":"AAaA,eAAO,MAAM,QAAQ,QAAQ,CAAC;AAY9B;;;;GAIG;AACH,wBAAgB,sBAAsB,CAAC,KAAK,EAAE,MAAM,GAAG,MAAM,CAK5D;AAED;;;;GAIG;AACH,wBAAgB,iBAAiB,CAC/B,UAAU,CAAC,EAAE,CAAC,GAAG,EAAE,MAAM,KAAK,IAAI,GACjC,CAAC,CAAC,GAAG,EAAE,MAAM,KAAK,IAAI,CAAC,GAAG,SAAS,CAGrC"}
|
|
@@ -0,0 +1,41 @@
|
|
|
1
|
+
// hazo_env/src/envsync/redact.ts — credential redaction for anything that leaves the process
|
|
2
|
+
//
|
|
3
|
+
// envsync handles two kinds of secret: connection strings (source_db /
|
|
4
|
+
// target_db are frequently full `postgres://user:pass@host/db` URIs, not bare
|
|
5
|
+
// database names) and .env.local values. Both routinely end up somewhere they
|
|
6
|
+
// shouldn't: progress lines are streamed to console.log by the CLI and the
|
|
7
|
+
// HTTP service, and the env diff is serialized into a JSON response body and
|
|
8
|
+
// rendered into the control page.
|
|
9
|
+
//
|
|
10
|
+
// Everything in this module is a pure string transform — apply it at the
|
|
11
|
+
// boundary (progress callback, HTTP response), never in the middle of the
|
|
12
|
+
// engine where the real value is still needed.
|
|
13
|
+
export const REDACTED = '***';
|
|
14
|
+
/** `scheme://user:pass@host` → `scheme://user:***@host` (username kept — it is
|
|
15
|
+
* useful for identifying which target an op ran against, the password is not). */
|
|
16
|
+
const URI_CREDENTIALS = /([a-zA-Z][a-zA-Z0-9+.-]*:\/\/)([^/\s:@]+):([^/\s@]*)@/g;
|
|
17
|
+
/** `?password=x` / `&sslpassword=x` in a connection URI's query string. */
|
|
18
|
+
const QUERY_PASSWORD = /([?&](?:password|passwd|pwd|sslpassword)=)[^&\s]*/gi;
|
|
19
|
+
/** libpq keyword/value form: `password=secret`, `password='se cret'`. */
|
|
20
|
+
const KEYWORD_PASSWORD = /\b(password\s*=\s*)(?:'[^']*'|"[^"]*"|\S+)/gi;
|
|
21
|
+
/**
|
|
22
|
+
* Strip credentials out of a connection string (or any text that may embed
|
|
23
|
+
* one — pg_dump/pg_restore echo the URI back in their own error output, so
|
|
24
|
+
* this is applied to whole progress lines, not just to config values).
|
|
25
|
+
*/
|
|
26
|
+
export function redactConnectionString(value) {
|
|
27
|
+
return value
|
|
28
|
+
.replace(URI_CREDENTIALS, (_m, scheme, user) => `${scheme}${user}:${REDACTED}@`)
|
|
29
|
+
.replace(QUERY_PASSWORD, `$1${REDACTED}`)
|
|
30
|
+
.replace(KEYWORD_PASSWORD, `$1${REDACTED}`);
|
|
31
|
+
}
|
|
32
|
+
/**
|
|
33
|
+
* Wrap a progress callback so every line it receives is credential-redacted.
|
|
34
|
+
* Returns undefined when there is no callback, so it can be passed straight
|
|
35
|
+
* through to runCmd's optional `onProgress`.
|
|
36
|
+
*/
|
|
37
|
+
export function redactingProgress(onProgress) {
|
|
38
|
+
if (!onProgress)
|
|
39
|
+
return undefined;
|
|
40
|
+
return (msg) => onProgress(redactConnectionString(msg));
|
|
41
|
+
}
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"service.d.ts","sourceRoot":"","sources":["../../src/envsync/service.ts"],"names":[],"mappings":"
|
|
1
|
+
{"version":3,"file":"service.d.ts","sourceRoot":"","sources":["../../src/envsync/service.ts"],"names":[],"mappings":"AAoCA,OAAO,IAAI,MAAM,WAAW,CAAC;AAiB7B,MAAM,WAAW,0BAA0B;IACzC,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,KAAK,CAAC,EAAE,MAAM,CAAC;CAChB;AAED,MAAM,WAAW,oBAAoB;IACnC,MAAM,EAAE,IAAI,CAAC,MAAM,CAAC;IACpB,KAAK,IAAI,OAAO,CAAC,IAAI,CAAC,CAAC;CACxB;AAqWD;;;;;;;;GAQG;AACH,wBAAgB,mBAAmB,CAAC,IAAI,GAAE,0BAA+B,GAAG,oBAAoB,CA8B/F"}
|
package/dist/envsync/service.js
CHANGED
|
@@ -25,11 +25,20 @@
|
|
|
25
25
|
// - Destructive ops (uploadDb, uploadFiles when confirm:true) go through
|
|
26
26
|
// the same assertConfirmed/assertNotProd guards the CLI uses — the HTTP
|
|
27
27
|
// layer adds no bypass.
|
|
28
|
+
// - `allowProd` is NEVER read from the request body: overriding the
|
|
29
|
+
// production guard is a server-side decision (`allow_prod = true` in
|
|
30
|
+
// [envsync]), so a caller cannot hand itself the override it is being
|
|
31
|
+
// guarded by. The control page has no "allow prod" control for the same
|
|
32
|
+
// reason.
|
|
33
|
+
// - Every `.env.local` diff is redacted (key + status, never values) before
|
|
34
|
+
// it is written into a response body or the control page — the diff is a
|
|
35
|
+
// complete dump of the target environment's secrets otherwise.
|
|
28
36
|
import http from 'node:http';
|
|
29
37
|
import fs from 'node:fs';
|
|
30
38
|
import path from 'node:path';
|
|
31
39
|
import { resolveEnvsyncConfig } from '../resolve/envsync.js';
|
|
32
40
|
import { downloadDb, uploadDb, downloadFiles, uploadFiles } from './engine.js';
|
|
41
|
+
import { redactEnvDiff } from './dotenv-merge.js';
|
|
33
42
|
import { withLock, EnvsyncLockError } from './lock.js';
|
|
34
43
|
import { pruneWorkDir } from './retention.js';
|
|
35
44
|
// Arbitrary high port in the range unlikely to collide with other local
|
|
@@ -175,7 +184,9 @@ async function handleDbUpload(req, res) {
|
|
|
175
184
|
try {
|
|
176
185
|
const result = await withLock(cfg.work_dir, 'upload_db', () => uploadDb(cfg, dumpPath, {
|
|
177
186
|
confirm: body['confirm'] === true,
|
|
178
|
-
allowProd
|
|
187
|
+
// allowProd comes from server config ONLY — body['allowProd'] is
|
|
188
|
+
// deliberately ignored (see the security model at the top).
|
|
189
|
+
allowProd: cfg.allowProd === true,
|
|
179
190
|
onProgress: progressLogger('upload_db'),
|
|
180
191
|
}));
|
|
181
192
|
sendJson(res, 200, result);
|
|
@@ -219,11 +230,14 @@ async function handleFilesRestore(req, res) {
|
|
|
219
230
|
// dry-run preview — that's a normal 200, not an error.
|
|
220
231
|
const result = await withLock(cfg.work_dir, 'upload_files', () => uploadFiles(cfg, archivePath, {
|
|
221
232
|
confirm: body['confirm'] === true,
|
|
222
|
-
|
|
233
|
+
// Server config only — see handleDbUpload.
|
|
234
|
+
allowProd: cfg.allowProd === true,
|
|
223
235
|
skipEnv: body['skipEnv'] === true,
|
|
236
|
+
pruneEnv: body['pruneEnv'] === true,
|
|
224
237
|
onProgress: progressLogger('upload_files'),
|
|
225
238
|
}));
|
|
226
|
-
|
|
239
|
+
// Values never leave the process: key + status only.
|
|
240
|
+
sendJson(res, 200, { ok: result.ok, diff: redactEnvDiff(result.diff) });
|
|
227
241
|
}
|
|
228
242
|
catch (err) {
|
|
229
243
|
sendEngineError(res, err);
|
|
@@ -246,6 +260,7 @@ function controlPageHtml() {
|
|
|
246
260
|
input[type="text"], input[type="password"] { padding: 0.4rem; width: 100%; box-sizing: border-box; margin-bottom: 0.5rem; }
|
|
247
261
|
pre { background: #8881; padding: 0.75rem; border-radius: 6px; overflow-x: auto; white-space: pre-wrap; word-break: break-word; }
|
|
248
262
|
label { font-weight: 600; }
|
|
263
|
+
.note { font-size: 0.8rem; opacity: 0.75; margin: 0.25rem 0 0; }
|
|
249
264
|
.row { display: flex; gap: 0.5rem; align-items: center; margin-bottom: 0.5rem; flex-wrap: wrap; }
|
|
250
265
|
</style>
|
|
251
266
|
</head>
|
|
@@ -271,9 +286,9 @@ function controlPageHtml() {
|
|
|
271
286
|
</div>
|
|
272
287
|
<div class="row">
|
|
273
288
|
<label><input type="checkbox" id="dbConfirm" /> confirm</label>
|
|
274
|
-
<label><input type="checkbox" id="dbAllowProd" /> allow prod</label>
|
|
275
289
|
<button id="dbUpload" type="button">Upload DB</button>
|
|
276
290
|
</div>
|
|
291
|
+
<p class="note">Production targets are refused unless <code>allow_prod = true</code> is set server-side in <code>[envsync]</code>.</p>
|
|
277
292
|
</section>
|
|
278
293
|
|
|
279
294
|
<section>
|
|
@@ -286,10 +301,11 @@ function controlPageHtml() {
|
|
|
286
301
|
</div>
|
|
287
302
|
<div class="row">
|
|
288
303
|
<label><input type="checkbox" id="filesConfirm" /> confirm</label>
|
|
289
|
-
<label><input type="checkbox" id="filesAllowProd" /> allow prod</label>
|
|
290
304
|
<label><input type="checkbox" id="filesSkipEnv" /> skip .env.local</label>
|
|
305
|
+
<label><input type="checkbox" id="filesPruneEnv" /> delete target-only .env.local keys</label>
|
|
291
306
|
<button id="filesRestore" type="button">Restore Files</button>
|
|
292
307
|
</div>
|
|
308
|
+
<p class="note">The <code>.env.local</code> diff is reported as key + status only — values are never sent over the wire.</p>
|
|
293
309
|
</section>
|
|
294
310
|
|
|
295
311
|
<pre id="output">(no output yet)</pre>
|
|
@@ -328,7 +344,6 @@ document.getElementById('dbUpload').addEventListener('click', function () {
|
|
|
328
344
|
call('POST', '/db/upload', {
|
|
329
345
|
dumpPath: document.getElementById('dumpPath').value,
|
|
330
346
|
confirm: document.getElementById('dbConfirm').checked,
|
|
331
|
-
allowProd: document.getElementById('dbAllowProd').checked,
|
|
332
347
|
});
|
|
333
348
|
});
|
|
334
349
|
document.getElementById('filesArchive').addEventListener('click', function () { call('POST', '/files/archive'); });
|
|
@@ -336,8 +351,8 @@ document.getElementById('filesRestore').addEventListener('click', function () {
|
|
|
336
351
|
call('POST', '/files/restore', {
|
|
337
352
|
archivePath: document.getElementById('archivePath').value,
|
|
338
353
|
confirm: document.getElementById('filesConfirm').checked,
|
|
339
|
-
allowProd: document.getElementById('filesAllowProd').checked,
|
|
340
354
|
skipEnv: document.getElementById('filesSkipEnv').checked,
|
|
355
|
+
pruneEnv: document.getElementById('filesPruneEnv').checked,
|
|
341
356
|
});
|
|
342
357
|
});
|
|
343
358
|
|
|
@@ -16,12 +16,27 @@ export interface EnvsyncConfig {
|
|
|
16
16
|
/** Shell command run after a successful pg_restore, if configured */
|
|
17
17
|
post_restore_cmd?: string;
|
|
18
18
|
/**
|
|
19
|
-
*
|
|
20
|
-
*
|
|
21
|
-
*
|
|
22
|
-
*
|
|
19
|
+
* Exact names guarded by assertNotProd — `prod_db_names` (comma-separated)
|
|
20
|
+
* from [envsync], always including source_db (the environment this config
|
|
21
|
+
* downloads FROM must never be overwritten by uploadDb/uploadFiles).
|
|
22
|
+
*
|
|
23
|
+
* This list is NOT the whole guard: assertNotProd also applies a name
|
|
24
|
+
* pattern (see prodDbPattern), because an exact-name list alone could never
|
|
25
|
+
* catch a staging→prod push.
|
|
23
26
|
*/
|
|
24
27
|
prodDbNames: string[];
|
|
28
|
+
/**
|
|
29
|
+
* Overrides guards.DEFAULT_PROD_NAME_PATTERN (`prod_db_pattern` in
|
|
30
|
+
* [envsync]). A case-insensitive regex source string; set it to an empty
|
|
31
|
+
* value to disable pattern matching and guard on prodDbNames alone.
|
|
32
|
+
*/
|
|
33
|
+
prodDbPattern?: string;
|
|
34
|
+
/**
|
|
35
|
+
* Server-side opt-in for writing to a production-looking target
|
|
36
|
+
* (`allow_prod` in [envsync], default false). The HTTP service reads
|
|
37
|
+
* allowProd from HERE ONLY — a request body can never set it.
|
|
38
|
+
*/
|
|
39
|
+
allowProd?: boolean;
|
|
25
40
|
/** Every KEY=value pair from [migrate.env_overrides], ${VAR}-expanded */
|
|
26
41
|
envOverrides: Record<string, string>;
|
|
27
42
|
}
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"envsync.d.ts","sourceRoot":"","sources":["../../src/resolve/envsync.ts"],"names":[],"mappings":"AASA,MAAM,WAAW,aAAa;IAC5B,gEAAgE;IAChE,SAAS,EAAE,MAAM,CAAC;IAClB,oEAAoE;IACpE,SAAS,EAAE,MAAM,CAAC;IAClB,kCAAkC;IAClC,KAAK,EAAE,MAAM,CAAC;IACd,0EAA0E;IAC1E,UAAU,EAAE,MAAM,CAAC;IACnB,oEAAoE;IACpE,QAAQ,EAAE,MAAM,CAAC;IACjB,oEAAoE;IACpE,IAAI,EAAE,MAAM,CAAC;IACb,yEAAyE;IACzE,eAAe,CAAC,EAAE,MAAM,CAAC;IACzB,qEAAqE;IACrE,gBAAgB,CAAC,EAAE,MAAM,CAAC;IAC1B
|
|
1
|
+
{"version":3,"file":"envsync.d.ts","sourceRoot":"","sources":["../../src/resolve/envsync.ts"],"names":[],"mappings":"AASA,MAAM,WAAW,aAAa;IAC5B,gEAAgE;IAChE,SAAS,EAAE,MAAM,CAAC;IAClB,oEAAoE;IACpE,SAAS,EAAE,MAAM,CAAC;IAClB,kCAAkC;IAClC,KAAK,EAAE,MAAM,CAAC;IACd,0EAA0E;IAC1E,UAAU,EAAE,MAAM,CAAC;IACnB,oEAAoE;IACpE,QAAQ,EAAE,MAAM,CAAC;IACjB,oEAAoE;IACpE,IAAI,EAAE,MAAM,CAAC;IACb,yEAAyE;IACzE,eAAe,CAAC,EAAE,MAAM,CAAC;IACzB,qEAAqE;IACrE,gBAAgB,CAAC,EAAE,MAAM,CAAC;IAC1B;;;;;;;;OAQG;IACH,WAAW,EAAE,MAAM,EAAE,CAAC;IACtB;;;;OAIG;IACH,aAAa,CAAC,EAAE,MAAM,CAAC;IACvB;;;;OAIG;IACH,SAAS,CAAC,EAAE,OAAO,CAAC;IACpB,yEAAyE;IACzE,YAAY,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC;CACtC;AAgBD;;;;;GAKG;AACH,wBAAgB,oBAAoB,IAAI,aAAa,GAAG,IAAI,CAuE3D"}
|
package/dist/resolve/envsync.js
CHANGED
|
@@ -38,6 +38,9 @@ export function resolveEnvsyncConfig() {
|
|
|
38
38
|
const keep_raw = section['keep'];
|
|
39
39
|
const pre_restore_cmd_raw = section['pre_restore_cmd'];
|
|
40
40
|
const post_restore_cmd_raw = section['post_restore_cmd'];
|
|
41
|
+
const prod_db_names_raw = section['prod_db_names'];
|
|
42
|
+
const prod_db_pattern_raw = section['prod_db_pattern'];
|
|
43
|
+
const allow_prod_raw = section['allow_prod'];
|
|
41
44
|
if (!source_db_raw || !target_db_raw || !owner_raw || !files_root_raw || !work_dir_raw)
|
|
42
45
|
return null;
|
|
43
46
|
// Resolve DATA_ROOT for ${DATA_ROOT} substitution in files_root, the same
|
|
@@ -53,6 +56,14 @@ export function resolveEnvsyncConfig() {
|
|
|
53
56
|
const source_db = expandEnvVars(source_db_raw, extraVars);
|
|
54
57
|
const keepParsed = keep_raw ? parseInt(keep_raw, 10) : NaN;
|
|
55
58
|
const keep = Number.isFinite(keepParsed) && keepParsed > 0 ? keepParsed : DEFAULT_KEEP;
|
|
59
|
+
// source_db is always guarded, plus whatever the operator listed.
|
|
60
|
+
const configuredProdNames = (prod_db_names_raw ?? '')
|
|
61
|
+
.split(',')
|
|
62
|
+
.map((name) => expandEnvVars(name.trim(), extraVars))
|
|
63
|
+
.filter(Boolean);
|
|
64
|
+
const prodDbNames = Array.from(new Set([source_db, ...configuredProdNames]));
|
|
65
|
+
const allowProd = allow_prod_raw === true ||
|
|
66
|
+
(typeof allow_prod_raw === 'string' && /^(1|true|yes|on)$/i.test(allow_prod_raw.trim()));
|
|
56
67
|
const envOverrides = {};
|
|
57
68
|
const overridesSection = config.getSection('migrate.env_overrides');
|
|
58
69
|
if (overridesSection) {
|
|
@@ -71,7 +82,9 @@ export function resolveEnvsyncConfig() {
|
|
|
71
82
|
keep,
|
|
72
83
|
pre_restore_cmd: pre_restore_cmd_raw ? expandEnvVars(pre_restore_cmd_raw, extraVars) : undefined,
|
|
73
84
|
post_restore_cmd: post_restore_cmd_raw ? expandEnvVars(post_restore_cmd_raw, extraVars) : undefined,
|
|
74
|
-
prodDbNames
|
|
85
|
+
prodDbNames,
|
|
86
|
+
prodDbPattern: prod_db_pattern_raw !== undefined ? prod_db_pattern_raw.trim() : undefined,
|
|
87
|
+
allowProd,
|
|
75
88
|
envOverrides,
|
|
76
89
|
};
|
|
77
90
|
}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "hazo_env",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.10.1",
|
|
4
4
|
"description": "Canonical environment resolver — typed env names, per-env DB/file/secret config, doctor and CLI for hazo apps",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "./dist/index.js",
|
|
@@ -38,13 +38,13 @@
|
|
|
38
38
|
"picocolors": "^1.1.1"
|
|
39
39
|
},
|
|
40
40
|
"peerDependencies": {
|
|
41
|
-
"hazo_core": "^
|
|
42
|
-
"hazo_config": "^2.4.
|
|
43
|
-
"hazo_connect": "^
|
|
44
|
-
"hazo_files": "^3.
|
|
45
|
-
"hazo_secure": "^1.
|
|
46
|
-
"hazo_audit": "^2.
|
|
47
|
-
"hazo_pdf": "^2.
|
|
41
|
+
"hazo_core": "^2.0.0",
|
|
42
|
+
"hazo_config": "^2.4.2",
|
|
43
|
+
"hazo_connect": "^4.0.0",
|
|
44
|
+
"hazo_files": "^3.2.0",
|
|
45
|
+
"hazo_secure": "^1.7.1",
|
|
46
|
+
"hazo_audit": "^2.2.0",
|
|
47
|
+
"hazo_pdf": "^2.2.0",
|
|
48
48
|
"react": "^18.0.0 || ^19.0.0",
|
|
49
49
|
"react-dom": "^18.0.0 || ^19.0.0",
|
|
50
50
|
"next": "^14.0.0 || ^16.0.0"
|
|
@@ -84,10 +84,10 @@
|
|
|
84
84
|
"@types/node": "^22.10.0",
|
|
85
85
|
"@types/react": "^19.0.0",
|
|
86
86
|
"@types/react-dom": "^19.0.0",
|
|
87
|
-
"hazo_core": "^
|
|
88
|
-
"hazo_config": "^2.4.
|
|
89
|
-
"hazo_connect": "^
|
|
90
|
-
"hazo_files": "^3.
|
|
87
|
+
"hazo_core": "^2.0.0",
|
|
88
|
+
"hazo_config": "^2.4.2",
|
|
89
|
+
"hazo_connect": "^4.0.0",
|
|
90
|
+
"hazo_files": "^3.2.0",
|
|
91
91
|
"next": "^16.0.10",
|
|
92
92
|
"react": "^19.0.0",
|
|
93
93
|
"react-dom": "^19.0.0",
|