envstash-cli 0.5.2 → 0.6.0

package/README.md

@@ -14,8 +14,8 @@ short form `envs`. They are the same program.
 
 ```bash
 envs login          # authorize this machine in the browser
-envs init           # create a project, named after your git remote
-envs push           # send your .env files up
+envs init           # create a project, named after your git remote, listing your .env files
+envs push           # send those files up
 ```
 
 On another machine, or a fresh clone:
@@ -24,22 +24,29 @@ On another machine, or a fresh clone:
 envs pull
 ```
 
+Day to day, when both sides have moved:
+
+```bash
+envs sync
+```
+
 ## Commands
 
-| Command   | What it does                                      |
-| --------- | ------------------------------------------------- |
-| `login`   | Authorize this machine with a browser device code |
-| `logout`  | Revoke this machine's token                       |
-| `whoami`  | Show the signed-in account                        |
-| `init`    | Create a project and link this repo to it         |
-| `link`    | Link this repo to a project that already exists   |
-| `list`    | List every project in your account                |
-| `status`  | Compare cloud files against local files           |
-| `diff`    | Show which keys differ, and how                   |
-| `pull`    | Write the cloud's version of each file to disk    |
-| `push`    | Send local files to the cloud                     |
-| `remove`  | Delete files from the cloud (local copies stay)   |
-| `version` | Show the CLI version and its environment          |
+| Command   | What it does                                         |
+| --------- | ---------------------------------------------------- |
+| `login`   | Authorize this machine with a browser device code    |
+| `logout`  | Revoke this machine's token                          |
+| `whoami`  | Show the signed-in account                           |
+| `init`    | Create a project and link this repo to it            |
+| `link`    | Link this repo to a project that already exists      |
+| `list`    | List every project in your account                   |
+| `status`  | Compare cloud files against local files              |
+| `diff`    | Show which keys differ, and how                      |
+| `pull`    | Write the cloud's version of each file to disk       |
+| `push`    | Send local files to the cloud                        |
+| `sync`    | Reconcile both directions at once, keeping every key |
+| `remove`  | Delete files from the cloud (local copies stay)      |
+| `version` | Show the CLI version and its environment             |
 
 Run `envs help <command>` for details on any one of them.
 
@@ -111,14 +118,109 @@ definition and therefore has no undo:
 - **`push` never deletes.** Keys in the cloud that are absent from your local
   file are left alone, so pushing a partial `.env` cannot destroy a teammate's
   key. Use `--prune` when you do want the cloud to mirror local exactly.
-- **`diff` masks values.** Use `--show-values` to reveal them.
+- **`push` lists new files before uploading them.** Any file that is not in the
+  cloud yet is shown, with its size, before the first upload. Files that
+  discovery found — not named on the command line and not listed in
+  `.envstash.json` — need a yes: `push` asks on a terminal, and anywhere else
+  (CI, a pipe) it stops unless you pass `--yes` (`-y`). Once a file is in the
+  cloud, later pushes of it go through without asking.
+- **`diff` masks values**, showing at most the first 4 characters and the
+  length, and nothing at all of a value 12 characters or shorter. Use
+  `--show-values` to reveal them.
+- **`sync` never deletes** in either direction, and never overwrites a value
+  that differs on both sides without being told which side wins.
 - **`remove` needs every file named** — there is no "remove everything"
   default — and it only touches the cloud copy; files on your machines stay.
-- Pulled files are written `0600`, and `~/.envstash/config.json` is `0600`
-  inside a `0700` directory.
+- Pulled files are written `0600` — replaced atomically, so a file that
+  already existed ends up `0600` too and a failed write leaves the old one —
+  and `~/.envstash/config.json` is written the same way, `0600` inside a `0700`
+  directory.
+- **Symlinks are never followed.** A tracked file, or any directory between it
+  and the repo root, that is a symbolic link is refused by every command that
+  reads or writes it, and discovery skips it — so a cloned repo cannot point
+  `.env.local` at `~/.ssh/id_rsa` and have `push` upload it.
+- **Some paths are never written**, whatever the server says: anything under
+  `.git/`, `node_modules/` or `.husky/`, and any `.envstash.json`. Files there
+  run code or redirect the CLI.
+- **Plain `http://` is refused** except for `localhost`, `*.localhost`,
+  `127.0.0.1` and `::1`, since every request carries your token.
+- Values containing `$` or a backtick are written single-quoted
+  (`PASS='p$ss'`), so bash, docker compose and dotenv-expand read them the
+  same way envstash does.
 
 Preview either direction with `--dry-run`.
 
+## `envs sync`
+
+`pull` and `push` each ask you to have already decided which side is right.
+Often neither is — a teammate added a key while you added a different one, and
+what you want is both. That is `sync`:
+
+```
+$ envs sync
+
+  Project demo-app  · https://www.envstash.co
+
+  ✓ .env.local        1 key pulled, 1 key pushed · 1 conflict
+      ~ SESSION_SECRET              (differs on both sides)
+  ✓ .env.staging      new from the cloud (4 keys)
+  = .env.production   already in sync
+
+  Synced 2 files · 5 keys down, 1 key up.
+  1 file left alone — the same key holds different values on each side.
+  See them with envs diff, then choose a side: envs sync --prefer local or envs sync --prefer cloud
+```
+
+| Situation                          | What `sync` does                    |
+| ---------------------------------- | ----------------------------------- |
+| File on disk, not in the cloud     | pushes it                           |
+| File in the cloud, not on disk     | pulls it                            |
+| Key on one side only               | copies it to the other side         |
+| Key on both sides, different value | **conflict** — left alone, reported |
+| Everything matches                 | nothing                             |
+
+**It never deletes.** Sync does not prune in either direction. There is no
+record of the last sync, so nothing can know whether a key that exists on one
+side only is one you just added or one a teammate removed — the same limit
+`envs status` acknowledges by saying `local-only` rather than `added`. Copying
+is the guess whose failure mode is a key coming back, which the next
+`envs status` shows you and `envs rm` undoes. Deleting is the guess whose
+failure mode is a secret that exists nowhere any more.
+
+**A conflict is never guessed at either.** When the same key holds different
+values on each side, both copies are left exactly as they are, the key is
+named, and `sync` exits **1** — so a hook or a CI job hears the difference
+between "sync ran" and "the repo is in sync". `--exit-zero` opts out, as it
+does for `status` and `diff`.
+
+Resolve conflicts by naming a winner:
+
+```bash
+envs sync --prefer local    # your value wins
+envs sync --prefer cloud    # the cloud's value wins
+```
+
+`--prefer` applies to conflicting keys only. Everything else still merges both
+ways, so `--prefer cloud` is not `pull --force`: your local-only keys still go
+up.
+
+An opaque file (see below) that differs on both sides is a whole-file conflict
+— half a certificate is not a certificate — so `--prefer` is the only way to
+settle one.
+
+Merging into a local file edits it in place: comments, blank lines and grouping
+survive, and only the keys that moved are rewritten. (`pull` still regenerates
+a file wholesale, because there the cloud's copy _is_ the answer.)
+
+Preview with `--dry-run`, narrow to one file by naming it, and reach your
+account-wide files with `--shared`:
+
+```bash
+envs sync --dry-run
+envs sync .env.local
+envs sync --shared .npmrc
+```
+
 ## Shared files
 
 Some credentials follow you rather than a repo — a global `.npmrc`, an AI-tool
@@ -169,7 +271,8 @@ hold:
 - it is not a document (`.md`, `.txt`, `.doc`, `.pdf`, and the like — notes,
   not credentials),
 - it is under the 1 MB blob limit, and
-- its name is one the sync commands accept (`[A-Za-z0-9._-]` per segment).
+- its name is one the sync commands accept (`[A-Za-z0-9._-]` per segment), and
+- neither it nor any directory above it (up to the repo root) is a symlink.
 
 Being git-ignored is the signal for "secret": it is the repo's own statement
 that a file is not for sharing through git, which is exactly the population

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "envstash-cli",
-  "version": "0.5.2",
+  "version": "0.6.0",
   "description": "Git-style CLI for EnvStash — keep your .env files in sync with your cloud account.",
   "keywords": [
     "env",

package/src/api.mjs

@@ -1,4 +1,4 @@
-import { CliError } from './ui.mjs'
+import { CliError, sanitize } from './ui.mjs'
 
 /** An error the server described in its JSON body. */
 export class ApiError extends CliError {
@@ -88,10 +88,12 @@ export async function request(
       )
     }
 
+    // Sanitized at the source as well as where errors are printed: this is the
+    // server's own wording, and anything that shows it should get it inert.
     throw new ApiError(
       response.status,
       payload.error ?? 'error',
-      payload.message,
+      sanitize(payload.message),
       response.status === 401 ? 'Run `envs login` to sign in again.' : undefined
     )
   }

package/src/atomic-write.mjs

@@ -0,0 +1,112 @@
+import { randomBytes } from 'node:crypto'
+import {
+  chmodSync,
+  closeSync,
+  fsyncSync,
+  openSync,
+  renameSync,
+  rmSync,
+  writeFileSync,
+} from 'node:fs'
+import { basename, dirname, join } from 'node:path'
+
+/**
+ * Replace `path` with `body` all at once, with exactly `mode`.
+ *
+ * Two problems with writing in place, both about files that hold credentials.
+ * `writeFileSync`'s `mode` only applies when it creates the file, so pulling
+ * over an existing 0644 `.env` left it 0644 however carefully the new contents
+ * were chosen. And a crash, a full disk or Ctrl-C part-way through left half a
+ * file — for `config.json`, every stored token gone.
+ *
+ * So: write a sibling temp file (same directory, so the rename cannot cross a
+ * filesystem), flush it, set the mode, and rename it over the target. The rename
+ * replaces a directory entry, so readers see the old file or the new one, never
+ * a torn one, and the result has the temp file's mode regardless of what was
+ * there. `wx` means the temp name is created fresh and never opened through
+ * anything already sitting at that path.
+ *
+ * `rename`, `sleep` and `platform` are seams for tests; callers pass only `mode`.
+ */
+export function writeFileAtomic(
+  path,
+  body,
+  { mode = 0o600, rename = renameSync, sleep = sleepSync, platform = process.platform } = {}
+) {
+  const temp = join(dirname(path), `.${basename(path)}.${randomBytes(6).toString('hex')}.tmp`)
+
+  try {
+    const fd = openSync(temp, 'wx', mode)
+
+    try {
+      writeFileSync(fd, body)
+      fsyncSync(fd)
+    } finally {
+      closeSync(fd)
+    }
+
+    try {
+      // Explicit, because `mode` on create is filtered through the umask.
+      chmodSync(temp, mode)
+    } catch {
+      // Windows and some network filesystems do not support POSIX modes.
+    }
+
+    renameWithRetry(temp, path, { rename, sleep, platform })
+  } catch (error) {
+    rmSync(temp, { force: true })
+    throw error
+  }
+}
+
+/**
+ * Codes Windows returns, transiently, when renaming over a file that an editor,
+ * indexer or antivirus scanner has open at that moment. The same retry
+ * write-file-atomic and graceful-fs do.
+ */
+const TRANSIENT_RENAME_CODES = new Set(['EPERM', 'EBUSY', 'EACCES'])
+
+/** Waits between the five attempts: about half a second in all. */
+const RENAME_BACKOFF_MS = [50, 100, 150, 200]
+
+/**
+ * Rename, retrying the transient Windows failures with a short backoff.
+ *
+ * Windows only: on POSIX those codes mean a real permission problem, and
+ * retrying would just delay the same error by half a second. Anything else, or
+ * the last attempt's failure, is thrown for the caller to clean up after.
+ */
+export function renameWithRetry(
+  from,
+  to,
+  { rename = renameSync, sleep = sleepSync, platform = process.platform } = {}
+) {
+  for (let attempt = 0; ; attempt += 1) {
+    try {
+      rename(from, to)
+      return
+    } catch (error) {
+      const retryable =
+        platform === 'win32' &&
+        TRANSIENT_RENAME_CODES.has(error?.code) &&
+        attempt < RENAME_BACKOFF_MS.length
+
+      if (!retryable) {
+        throw error
+      }
+
+      sleep(RENAME_BACKOFF_MS[attempt])
+    }
+  }
+}
+
+/**
+ * Block for `ms` without spinning. Synchronous on purpose: every caller of
+ * `writeFileAtomic` is synchronous (the config writes behind `saveToken` and
+ * `clearToken` are called from sync code throughout), and making the whole
+ * chain async for a wait that only happens on a Windows file lock is not worth
+ * it. `Atomics.wait` on a private buffer that nothing ever notifies is a sleep.
+ */
+function sleepSync(ms) {
+  Atomics.wait(new Int32Array(new SharedArrayBuffer(4)), 0, 0, ms)
+}

package/src/commands/diff.mjs

@@ -3,7 +3,7 @@ import { formatSize } from '../blob.mjs'
 import { cloudKinds, loadLocalFiles, resolveContext, resolveFileNames } from '../context.mjs'
 import { entriesToMap } from '../env-file.mjs'
 import { compareProject, driftExitCode, FILE_STATE, KEY_STATE } from '../status.mjs'
-import { maskValue, out, style } from '../ui.mjs'
+import { maskValue, out, sanitize, style } from '../ui.mjs'
 
 /**
  * `envs diff` — the same comparison as `status`, with values.
@@ -102,10 +102,11 @@ export async function diff(flags, args) {
 }
 
 function renderKey(entry, local, cloud, showValues) {
-  const reveal = value => (showValues ? value : maskValue(value))
+  const reveal = value => (showValues ? showValue(value) : maskValue(value))
+  const key = sanitize(entry.key)
 
   if (entry.state === KEY_STATE.MODIFIED) {
-    out(`    ${style.yellow('~')} ${entry.key}`)
+    out(`    ${style.yellow('~')} ${key}`)
     out(
       `        ${style.red('cloud')} ${cloud ? reveal(cloud.get(entry.key) ?? '') : style.dim('•'.repeat(8))}`
     )
@@ -115,18 +116,28 @@ function renderKey(entry, local, cloud, showValues) {
 
   if (entry.state === KEY_STATE.LOCAL_ONLY) {
     out(
-      `    ${style.green('+')} ${entry.key}  ${reveal(local.get(entry.key) ?? '')} ${style.dim('(local only)')}`
+      `    ${style.green('+')} ${key}  ${reveal(local.get(entry.key) ?? '')} ${style.dim('(local only)')}`
     )
     return
   }
 
   out(
-    `    ${style.red('-')} ${entry.key}  ${
+    `    ${style.red('-')} ${key}  ${
       cloud ? reveal(cloud.get(entry.key) ?? '') : style.dim('•'.repeat(8))
     } ${style.dim('(cloud only)')}`
   )
 }
 
+/**
+ * A revealed value, one line and inert. Values come from the server as well as
+ * the disk, so escape sequences are stripped; line breaks in a multi-line value
+ * (a PEM key) are shown as `\n` rather than breaking the diff's layout — or
+ * forging a line of it.
+ */
+function showValue(value) {
+  return sanitize(value.replace(/\r/g, '\\r').replace(/\n/g, '\\n'))
+}
+
 function describeFile(file) {
   if (file.state === FILE_STATE.CLOUD_ONLY) {
     return 'in the cloud, not on disk'

package/src/commands/login.mjs

@@ -14,7 +14,7 @@ import {
 } from '../config.mjs'
 import { DEV_PORTS } from '../dev-host.mjs'
 import { isLinkChosenHost, resolveHostFor, resolveLoginHost } from '../session.mjs'
-import { CliError, out, style } from '../ui.mjs'
+import { CliError, out, sanitize, style } from '../ui.mjs'
 
 /**
  * `envs login` — the OAuth device flow.
@@ -51,7 +51,7 @@ export async function login(flags) {
   const verificationUrl = safeVerificationUrl(started.verificationUrlComplete, host)
 
   out()
-  out(`  Your code:  ${style.bold(style.cyan(started.userCode))}`)
+  out(`  Your code:  ${style.bold(style.cyan(sanitize(started.userCode)))}`)
   out()
   // The URL on its own line, in the same visible cyan as the code above it:
   // blue vanishes on dark terminals, and a line that is only the URL is what
@@ -98,7 +98,9 @@ export async function login(flags) {
   // command in every other repo talking to it.
   saveToken(host, token, who.email, { makeDefault: !fromLink })
 
-  out(`  ${style.green('✓')} Signed in to ${style.bold(host)} as ${style.bold(who.email)}`)
+  out(
+    `  ${style.green('✓')} Signed in to ${style.bold(host)} as ${style.bold(sanitize(who.email))}`
+  )
   out()
 
   return 0
@@ -110,8 +112,7 @@ export async function login(flags) {
  * a CliError with the server's own wording.
  */
 async function pollForToken(host, started) {
-  const intervalMs = Math.max(1, started.intervalSeconds ?? 2) * 1000
-  const deadline = new Date(started.expiresAt).getTime()
+  const { intervalMs, deadline } = pollSchedule(started)
 
   while (Date.now() < deadline) {
     await sleep(intervalMs)
@@ -128,6 +129,29 @@ async function pollForToken(host, started) {
   })
 }
 
+/** The server's codes live 15 minutes; nothing legitimate needs longer. */
+export const MAX_POLL_MS = 15 * 60 * 1000
+
+const MAX_INTERVAL_MS = 30 * 1000
+
+/**
+ * How often and until when to poll, from the server's answer — bounded here,
+ * because the server chose both numbers. An `expiresAt` a year out (or an
+ * unparseable one) would otherwise leave a terminal polling for a year, and an
+ * `intervalSeconds` of a million would leave it asleep.
+ */
+export function pollSchedule(started, now = Date.now()) {
+  const seconds = Number(started.intervalSeconds ?? 2)
+  const intervalMs = Math.min(
+    MAX_INTERVAL_MS,
+    Math.max(1000, Number.isFinite(seconds) ? seconds * 1000 : 2000)
+  )
+  const cap = now + MAX_POLL_MS
+  const expires = new Date(started.expiresAt).getTime()
+
+  return { intervalMs, deadline: Number.isFinite(expires) ? Math.min(expires, cap) : cap }
+}
+
 /** Name the device so the approval screen and the token list are recognisable. */
 function defaultLabel() {
   try {

package/src/commands/misc.mjs

@@ -12,7 +12,7 @@ import {
   writeLink,
 } from '../project.mjs'
 import { requireHostToken, resolveHostFor, resolveHostSource } from '../session.mjs'
-import { out, pad, pluralize, style } from '../ui.mjs'
+import { out, pad, pluralize, sanitize, style } from '../ui.mjs'
 import { announceIgnoredSecretDir } from './push.mjs'
 
 /**
@@ -87,7 +87,9 @@ export async function whoami(flags) {
   }
 
   out()
-  out(`  ${style.bold(who.email)}${who.name ? style.dim(` (${who.name})`) : ''}`)
+  out(
+    `  ${style.bold(sanitize(who.email))}${who.name ? style.dim(` (${sanitize(who.name)})`) : ''}`
+  )
   out(`  ${style.dim(host)}`)
   out()
 
@@ -112,12 +114,12 @@ export async function list(flags) {
     return 0
   }
 
-  const width = Math.max(...projects.map(project => project.name.length)) + 4
+  const width = Math.max(...projects.map(project => sanitize(project.name).length)) + 4
 
   for (const project of projects) {
     out(
-      `  ${pad(style.bold(project.name), width)}${style.dim(
-        `${pluralize(project.fileCount, 'file')}, ${pluralize(project.secretCount, 'key')}`
+      `  ${pad(style.bold(sanitize(project.name)), width)}${style.dim(
+        `${pluralize(Number(project.fileCount) || 0, 'file')}, ${pluralize(Number(project.secretCount) || 0, 'key')}`
       )}`
     )
   }
@@ -194,7 +196,7 @@ export async function link(flags, args) {
     if (projects?.length) {
       out(`  ${style.dim('Available projects:')}`)
       for (const project of projects) {
-        out(`    ${style.dim('·')} ${project.name}`)
+        out(`    ${style.dim('·')} ${sanitize(project.name)}`)
       }
       out()
     }

package/src/commands/pull.mjs

@@ -1,6 +1,3 @@
-import { mkdirSync, writeFileSync } from 'node:fs'
-import { dirname } from 'node:path'
-
 import { api, ApiError } from '../api.mjs'
 import { formatSize } from '../blob.mjs'
 import {
@@ -11,7 +8,7 @@ import {
   SHARED_SCOPE,
 } from '../context.mjs'
 import { serializeEnv } from '../env-file.mjs'
-import { resolveInsideRepo } from '../repo-path.mjs'
+import { writeSecretFile } from '../local-write.mjs'
 import { compareProject, FILE_STATE } from '../status.mjs'
 import { CliError, out, pluralize, style } from '../ui.mjs'
 
@@ -67,6 +64,7 @@ export async function pull(flags, args) {
     link: context.link,
     cloudFiles,
     requested: args,
+    sink: 'write',
   }).filter(name => cloudFiles.some(file => file.name === name))
 
   if (names.length === 0) {
@@ -126,20 +124,12 @@ export async function pull(flags, args) {
       ? `(file, ${formatSize(body.length)})`
       : `(${pluralize(response.entries?.length ?? 0, 'key')})`
 
-    // Re-checked at the sink, not only where the name entered. This is the call
-    // that creates directories and writes bytes, so it is the one that has to
-    // be certain the path is inside the repo.
-    const path = resolveInsideRepo(context.repoRoot, file.name)
-
     if (flags['dry-run']) {
       out(`  ${style.dim('would write')} ${file.name} ${style.dim(detail)}`)
       continue
     }
 
-    mkdirSync(dirname(path), { recursive: true })
-    // 0600: a pulled secret holds live credentials and has no business being
-    // readable by other accounts on a shared machine.
-    writeFileSync(path, body, { mode: 0o600 })
+    writeSecretFile(context.repoRoot, file.name, body)
     written += 1
 
     out(`  ${style.green('✓')} ${file.name} ${style.dim(detail)}`)