@dotenvx/dotenvx 1.36.0 → 1.38.0

package/CHANGELOG.md

@@ -2,7 +2,22 @@
 
 All notable changes to this project will be documented in this file. See [standard-version](https://github.com/conventional-changelog/standard-version) for commit guidelines.
 
-[Unreleased](https://github.com/dotenvx/dotenvx/compare/v1.36.0...main)
+[Unreleased](https://github.com/dotenvx/dotenvx/compare/v1.38.0...main)
+
+## [1.38.0](https://github.com/dotenvx/dotenvx/compare/v1.37.0...v1.38.0)
+
+### Changed
+
+* Command substitution failures no longer halt further processing of keys in a .env file ([#533](https://github.com/dotenvx/dotenvx/pull/533))
+* A more helpful error is raised if a command substitution failure occurs ([#533](https://github.com/dotenvx/dotenvx/pull/533))
+
+## [1.37.0](https://github.com/dotenvx/dotenvx/compare/v1.36.0...v1.37.0)
+
+### Added
+
+* add `dotenvx rotate` command 🎉 ([#530](https://github.com/dotenvx/dotenvx/pull/530))
+
+also: [our whitepaper](https://dotenvx.com/dotenvx.pdf) is released as a draft.
 
 ## [1.36.0](https://github.com/dotenvx/dotenvx/compare/v1.35.0...v1.36.0)
 

package/README.md

@@ -6,7 +6,7 @@
 * multi-environment
 * encrypted envs
 
-[Read the whitepaper](https://dotenvx.com/dotenvx.pdf)
+[Read the whitepaper](https://dotenvx.com/dotenvx.pdf?v=README)
 
  
 
@@ -1685,7 +1685,117 @@ More examples
   ```
 
   </details>
+* <details><summary>`rotate`</summary><br>
 
+  Rotate public/private keys for `.env` file and re-encrypt all encrypted values.
+
+  ```sh
+  $ echo "HELLO=World" > .env
+  $ dotenvx encrypt
+  ✔ encrypted (.env)
+  $ dotenvx rotate
+  ✔ rotated (.env)
+  ```
+
+  </details>
+* <details><summary>`rotate -f`</summary><br>
+
+  Rotate public/private keys for a specified encrypted `.env` file and re-encrypt all encrypted values.
+
+  ```sh
+  $ echo "HELLO=World" > .env
+  $ echo "HELLO=Production" > .env.production
+
+  $ dotenvx encrypt -f .env.production
+  ✔ encrypted (.env.production)
+  $ dotenvx rotate -f .env.production
+  ✔ rotated (.env.production)
+  ```
+
+  </details>
+* <details><summary>`rotate -fk`</summary><br>
+
+  Specify path to `.env.keys`. This is useful with monorepos.
+
+  ```sh
+  $ mkdir -p apps/app1
+  $ echo "HELLO=World" > apps/app1/.env
+
+  $ dotenvx encrypt -fk .env.keys -f apps/app1/.env
+  ✔ encrypted (apps/app1/.env)
+  $ dotenvx rotate -fk .env.keys -f apps/app1/.env
+  ✔ rotated (apps/app1/.env)
+  ```
+
+  </details>
+* <details><summary>`rotate -k`</summary><br>
+
+  Rotate the contents of a specified key inside an encrypted `.env` file.
+
+  ```sh
+  $ echo "HELLO=World\nHOLA=Mundo" > .env
+  $ dotenvx encrypt
+  ✔ encrypted (.env)
+  $ dotenvx rotate -k HELLO
+  ✔ rotated (.env)
+  ```
+
+  Even specify a glob pattern.
+
+  ```sh
+  $ echo "HELLO=World\nHOLA=Mundo" > .env
+  $ dotenvx encrypt
+  ✔ encrypted (.env)
+  $ dotenvx rotate -k "HE*"
+  ✔ rotated (.env)
+  ```
+
+  </details>
+* <details><summary>`rotate -ek`</summary><br>
+
+  Rotate the encrypted contents inside an encrypted `.env` file except for an exluded key.
+
+  ```sh
+  $ echo "HELLO=World\nHOLA=Mundo" > .env
+  $ dotenvx encrypt
+  ✔ encrypted (.env)
+  $ dotenvx rotate -ek HOLA
+  ✔ rotated (.env)
+  ```
+
+  Even specify a glob pattern.
+
+  ```sh
+  $ echo "HELLO=World\nHOLA=Mundo" > .env
+  $ dotenvx encrypt
+  ✔ encrypted (.env)
+  $ dotenvx rotate -ek "HO*"
+  ✔ rotated (.env)
+  ```
+
+  </details>
+* <details><summary>`rotate --stdout`</summary><br>
+
+  Rotate the contents of an encrypted `.env` file and send to stdout.
+
+  ```sh
+  $ dotenvx rotate --stdout
+  #/-------------------[DOTENV_PUBLIC_KEY]--------------------/
+  #/            public-key encryption for .env files          /
+  #/       [how it works](https://dotenvx.com/encryption)     /
+  #/----------------------------------------------------------/
+  DOTENV_PUBLIC_KEY="034af93e93708b994c10f236c96ef88e47291066946cce2e8d98c9e02c741ced45"
+  # .env
+  HELLO="encrypted:12345"
+  ```
+
+  or send to a file:
+
+  ```sh
+  $ dotenvx rotate --stdout > somefile.txt
+  ```
+
+  </details>
 * <details><summary>`help`</summary><br>
 
   Output help for `dotenvx`.
@@ -2134,6 +2244,22 @@ More examples
 
 ## FAQ
 
+#### How does encryption work?
+
+Dotenvx uses Elliptic Curve Integrated Encryption Scheme (ECIES) to encrypt each secret with a unique ephemeral key, while ensuring it can be decrypted using a long-term private key.
+
+When you initialize encryption, a DOTENV_PUBLIC_KEY (encryption key) and DOTENV_PRIVATE_KEY (decryption key) are generated. The DOTENV_PUBLIC_KEY is used to encrypt secrets, and the DOTENV_PRIVATE_KEY is securely stored in your cloud secrets manager or .env.keys file.
+
+Your encrypted .env file is then safely committed to code. Even if the file is exposed, secrets remain protected since decryption requires the separate DOTENV_PRIVATE_KEY, which is never stored alongside it. Read [the whitepaper](https://dotenvx.com/dotenvx.pdf?v=README) for more details.
+
+#### Is it safe to commit an encrypted .env file to code?
+
+Yes. Dotenvx encrypts secrets using AES-256 with ephemeral keys, ensuring that even if the encrypted .env file is exposed, its contents remain secure. The encryption keys themselves are protected using Secp256k1 elliptic curve cryptography, which is widely used for secure key exchange in technologies like Bitcoin.
+
+This means that every secret in the .env file is encrypted with a unique AES-256 key, and that key is further encrypted using a public key (Secp256k1). Even if an attacker obtains the encrypted .env file, they would still need the corresponding private key—stored separately in a secrets manager—to decrypt anything.
+
+Breaking this encryption would require brute-forcing both AES-256 and elliptic curve cryptography, which is computationally infeasible with current technology. Read [the whitepaper](https://dotenvx.com/dotenvx.pdf?v=README) for more details.
+
 #### Why am I getting the error `node: .env: not found`?
 
 You are using Node 20 or greater and it adds a differing implementation of `--env-file` flag support. Rather than warn on a missing `.env` file (like dotenv has historically done), it raises an error: `node: .env: not found`.

package/package.json

@@ -1,5 +1,5 @@
 {
-  "version": "1.36.0",
+  "version": "1.38.0",
   "name": "@dotenvx/dotenvx",
   "description": "a better dotenv–from the creator of `dotenv`",
   "author": "@motdotla",

package/src/cli/actions/rotate.js

@@ -0,0 +1,83 @@
+const fsx = require('./../../lib/helpers/fsx')
+const { logger } = require('./../../shared/logger')
+
+const Rotate = require('./../../lib/services/rotate')
+
+const catchAndLog = require('../../lib/helpers/catchAndLog')
+const isIgnoringDotenvKeys = require('../../lib/helpers/isIgnoringDotenvKeys')
+
+function rotate () {
+  const options = this.opts()
+  logger.debug(`options: ${JSON.stringify(options)}`)
+
+  const envs = this.envs
+
+  // stdout - should not have a try so that exit codes can surface to stdout
+  if (options.stdout) {
+    const {
+      processedEnvs
+    } = new Rotate(envs, options.key, options.excludeKey, options.envKeysFile).run()
+
+    for (const processedEnv of processedEnvs) {
+      console.log(processedEnv.envSrc)
+      console.log('')
+      console.log(processedEnv.envKeysSrc)
+    }
+    process.exit(0) // exit early
+  } else {
+    try {
+      const {
+        processedEnvs,
+        changedFilepaths,
+        unchangedFilepaths
+      } = new Rotate(envs, options.key, options.excludeKey, options.envKeysFile).run()
+
+      for (const processedEnv of processedEnvs) {
+        logger.verbose(`rotating ${processedEnv.envFilepath} (${processedEnv.filepath})`)
+        if (processedEnv.error) {
+          if (processedEnv.error.code === 'MISSING_ENV_FILE') {
+            logger.warn(processedEnv.error.message)
+            logger.help(`? add one with [echo "HELLO=World" > ${processedEnv.envFilepath}] and re-run [dotenvx rotate]`)
+          } else {
+            logger.warn(processedEnv.error.message)
+            if (processedEnv.error.help) {
+              logger.help(processedEnv.error.help)
+            }
+          }
+        } else if (processedEnv.changed) {
+          fsx.writeFileX(processedEnv.filepath, processedEnv.envSrc)
+          fsx.writeFileX(processedEnv.envKeysFilepath, processedEnv.envKeysSrc)
+
+          logger.verbose(`rotated ${processedEnv.envFilepath} (${processedEnv.filepath})`)
+        } else {
+          logger.verbose(`no changes ${processedEnv.envFilepath} (${processedEnv.filepath})`)
+        }
+      }
+
+      if (changedFilepaths.length > 0) {
+        logger.success(`✔ rotated (${changedFilepaths.join(',')})`)
+      } else if (unchangedFilepaths.length > 0) {
+        logger.info(`no changes (${unchangedFilepaths})`)
+      } else {
+        // do nothing - scenario when no .env files found
+      }
+
+      for (const processedEnv of processedEnvs) {
+        if (processedEnv.privateKeyAdded) {
+          logger.success(`✔ key added to .env.keys (${processedEnv.privateKeyName})`)
+
+          if (!isIgnoringDotenvKeys()) {
+            logger.help('⮕  next run [dotenvx ext gitignore --pattern .env.keys] to gitignore .env.keys')
+          }
+
+          logger.help(`⮕  next run [${processedEnv.privateKeyName}='${processedEnv.privateKey}' dotenvx get] to test decryption locally`)
+        }
+      }
+    } catch (error) {
+      catchAndLog(error)
+      process.exit(1)
+    }
+  }
+}
+
+module.exports = rotate

package/src/cli/dotenvx.js

@@ -156,6 +156,20 @@ program.command('ls')
   .option('-ef, --exclude-env-file <excludeFilenames...>', 'path(s) to exclude from your env file(s) (default: none)')
   .action(lsAction)
 
+// dotenvx rotate
+const rotateAction = require('./actions/rotate')
+program.command('rotate')
+  .description('rotate keypair(s) and re-encrypt .env file(s)')
+  .option('-f, --env-file <paths...>', 'path(s) to your env file(s)', collectEnvs('envFile'), [])
+  .option('-fk, --env-keys-file <path>', 'path to your .env.keys file (default: same path as your env file)')
+  .option('-k, --key <keys...>', 'keys(s) to encrypt (default: all keys in file)')
+  .option('-ek, --exclude-key <excludeKeys...>', 'keys(s) to exclude from encryption (default: none)')
+  .option('--stdout', 'send to stdout')
+  .action(function (...args) {
+    this.envs = envs
+    rotateAction.apply(this, args)
+  })
+
 // dotenvx help
 program.command('help [command]')
   .description('display help for command')

package/src/lib/helpers/append.js

@@ -0,0 +1,61 @@
+const quotes = require('./quotes')
+const dotenvParse = require('./dotenvParse')
+const escapeForRegex = require('./escapeForRegex')
+const escapeDollarSigns = require('./escapeDollarSigns')
+
+function append (src, key, appendValue) {
+  let output
+  let newPart = ''
+
+  const parsed = dotenvParse(src, true) // skip expanding \n
+  const _quotes = quotes(src)
+  if (Object.prototype.hasOwnProperty.call(parsed, key)) {
+    const quote = _quotes[key]
+    const originalValue = parsed[key]
+
+    newPart += `${key}=${quote}${originalValue},${appendValue}${quote}`
+
+    const escapedOriginalValue = escapeForRegex(originalValue)
+
+    // conditionally enforce end of line
+    let enforceEndOfLine = ''
+    if (escapedOriginalValue === '') {
+      enforceEndOfLine = '$' // EMPTY scenario
+    }
+
+    const currentPart = new RegExp(
+      '^' + // start of line
+      '(\\s*)?' + // spaces
+      '(export\\s+)?' + // export
+      key + // KEY
+      '\\s*=\\s*' + // spaces (KEY = value)
+      '["\'`]?' + // open quote
+      escapedOriginalValue + // escaped value
+      '["\'`]?' + // close quote
+      enforceEndOfLine
+      ,
+      'gm' // (g)lobal (m)ultiline
+    )
+
+    const saferInput = escapeDollarSigns(newPart) // cleanse user inputted capture groups ($1, $2 etc)
+
+    // $1 preserves spaces
+    // $2 preserves export
+    output = src.replace(currentPart, `$1$2${saferInput}`)
+  } else {
+    newPart += `${key}="${appendValue}"`
+
+    // append
+    if (src.endsWith('\n')) {
+      newPart = newPart + '\n'
+    } else {
+      newPart = '\n' + newPart
+    }
+
+    output = src + newPart
+  }
+
+  return output
+}
+
+module.exports = append

package/src/lib/helpers/errors.js

@@ -8,6 +8,7 @@ class Errors {
     this.key = options.key
     this.privateKey = options.privateKey
     this.privateKeyName = options.privateKeyName
+    this.command = options.command
 
     this.message = options.message
   }
@@ -84,6 +85,17 @@ class Errors {
     e.code = code
     return e
   }
+
+  commandSubstitutionFailed () {
+    const code = 'COMMAND_SUBSTITUTION_FAILED'
+    const message = `[${code}] could not eval ${this.key} containing command '${this.command}': ${this.message}`
+    const help = `[${code}] https://github.com/dotenvx/dotenvx/issues/532`
+
+    const e = new Error(message)
+    e.code = code
+    e.help = help
+    return e
+  }
 }
 
 module.exports = Errors

package/src/lib/helpers/evalKeyValue.js

@@ -0,0 +1,23 @@
+const { execSync } = require('child_process')
+const chomp = require('./chomp')
+const Errors = require('./errors')
+
+function evalKeyValue (key, value, processEnv, runningParsed) {
+  // Match everything between the outermost $() using a regex with non-capturing groups
+  const matches = value.match(/\$\(([^)]+(?:\)[^(]*)*)\)/g) || []
+  return matches.reduce((newValue, match) => {
+    const command = match.slice(2, -1) // Extract command by removing $() wrapper
+    let result
+
+    try {
+      result = execSync(command, { env: { ...processEnv, ...runningParsed } }).toString() // execute command (including runningParsed)
+    } catch (e) {
+      throw new Errors({ key, command, message: e.message.trim() }).commandSubstitutionFailed()
+    }
+
+    result = chomp(result) // chomp it
+    return newValue.replace(match, result) // Replace match with result
+  }, value)
+}
+
+module.exports = evalKeyValue

package/src/lib/helpers/parse.js

@@ -1,7 +1,6 @@
-const chomp = require('./chomp')
 const decryptKeyValue = require('./decryptKeyValue')
+const evalKeyValue = require('./evalKeyValue')
 const resolveEscapeSequences = require('./resolveEscapeSequences')
-const { execSync } = require('child_process')
 
 class Parse {
   static LINE = /(?:^|^)\s*(?:export\s+)?([\w.-]+)(?:\s*=\s*?|:\s+?)(\s*'(?:\\'|[^'])*'|\s*"(?:\\"|[^"])*"|\s*`(?:\\`|[^`])*`|[^#\r\n]+)?\s*(?:#.*)?(?:$|$)/mg
@@ -49,7 +48,12 @@ class Parse {
       let evaled = false
       if (quote !== "'" && (!this.inProcessEnv(key) || this.processEnv[key] === this.parsed[key])) {
         const priorEvaled = this.parsed[key]
-        this.parsed[key] = this.eval(priorEvaled)
+        // eval
+        try {
+          this.parsed[key] = this.eval(key, priorEvaled)
+        } catch (e) {
+          this.errors.push(e)
+        }
         if (priorEvaled !== this.parsed[key]) {
           evaled = true
         }
@@ -133,14 +137,8 @@ class Parse {
     return decryptKeyValue(key, value, this.privateKeyName, this.privateKey)
   }
 
-  eval (value) {
-    // Match everything between the outermost $() using a regex with non-capturing groups
-    const matches = value.match(/\$\(([^)]+(?:\)[^(]*)*)\)/g) || []
-    return matches.reduce((newValue, match) => {
-      const command = match.slice(2, -1) // Extract command by removing $() wrapper
-      const result = chomp(execSync(command, { env: { ...this.processEnv, ...this.runningParsed } }).toString()) // execute command (including runningParsed)
-      return newValue.replace(match, result) // Replace match with result
-    }, value)
+  eval (key, value) {
+    return evalKeyValue(key, value, this.processEnv, this.runningParsed)
   }
 
   expand (value) {

package/src/lib/services/rotate.js

@@ -0,0 +1,160 @@
+const fsx = require('./../helpers/fsx')
+const path = require('path')
+const picomatch = require('picomatch')
+
+const TYPE_ENV_FILE = 'envFile'
+
+const Errors = require('./../helpers/errors')
+const guessPrivateKeyName = require('./../helpers/guessPrivateKeyName')
+const guessPublicKeyName = require('./../helpers/guessPublicKeyName')
+const encryptValue = require('./../helpers/encryptValue')
+const isEncrypted = require('./../helpers/isEncrypted')
+const dotenvParse = require('./../helpers/dotenvParse')
+const replace = require('./../helpers/replace')
+const append = require('./../helpers/append')
+const detectEncoding = require('./../helpers/detectEncoding')
+const determineEnvs = require('./../helpers/determineEnvs')
+const findPrivateKey = require('./../helpers/findPrivateKey')
+const decryptKeyValue = require('./../helpers/decryptKeyValue')
+const keypair = require('./../helpers/keypair')
+
+class Rotate {
+  constructor (envs = [], key = [], excludeKey = [], envKeysFilepath = null) {
+    this.envs = determineEnvs(envs, process.env)
+    this.key = key
+    this.excludeKey = excludeKey
+    this.envKeysFilepath = envKeysFilepath
+
+    this.processedEnvs = []
+    this.changedFilepaths = new Set()
+    this.unchangedFilepaths = new Set()
+
+    this.envKeysSources = {}
+  }
+
+  run () {
+    // example
+    // envs [
+    //   { type: 'envFile', value: '.env' }
+    // ]
+
+    this.keys = this._keys()
+    const excludeKeys = this._excludeKeys()
+
+    this.exclude = picomatch(excludeKeys)
+    this.include = picomatch(this.keys, { ignore: excludeKeys })
+
+    for (const env of this.envs) {
+      if (env.type === TYPE_ENV_FILE) {
+        this._rotateEnvFile(env.value)
+      }
+    }
+
+    return {
+      processedEnvs: this.processedEnvs,
+      changedFilepaths: [...this.changedFilepaths],
+      unchangedFilepaths: [...this.unchangedFilepaths]
+    }
+  }
+
+  _rotateEnvFile (envFilepath) {
+    const row = {}
+    row.keys = []
+    row.type = TYPE_ENV_FILE
+
+    const filepath = path.resolve(envFilepath)
+    row.filepath = filepath
+    row.envFilepath = envFilepath
+
+    try {
+      const encoding = this._detectEncoding(filepath)
+      let envSrc = fsx.readFileX(filepath, { encoding })
+      const envParsed = dotenvParse(envSrc)
+
+      const publicKeyName = guessPublicKeyName(envFilepath)
+      const privateKeyName = guessPrivateKeyName(envFilepath)
+      const existingPrivateKey = findPrivateKey(envFilepath, this.envKeysFilepath)
+
+      let envKeysFilepath = path.join(path.dirname(filepath), '.env.keys')
+      if (this.envKeysFilepath) {
+        envKeysFilepath = path.resolve(this.envKeysFilepath)
+      }
+      const keysEncoding = this._detectEncoding(envKeysFilepath)
+
+      row.envKeysFilepath = envKeysFilepath
+      this.envKeysSources[envKeysFilepath] ||= fsx.readFileX(envKeysFilepath, { encoding: keysEncoding })
+      let envKeysSrc = this.envKeysSources[envKeysFilepath]
+
+      // new keypair
+      const nkp = keypair() // generates a fresh keypair in memory
+      const newPublicKey = nkp.publicKey
+      const newPrivateKey = nkp.privateKey
+
+      // .env
+      envSrc = replace(envSrc, publicKeyName, newPublicKey) // replace publicKey
+      row.changed = true // track change
+      for (const [key, value] of Object.entries(envParsed)) { // re-encrypt each individual key
+        // key excluded - don't re-encrypt it
+        if (this.exclude(key)) {
+          continue
+        }
+
+        // key effectively excluded (by not being in the list of includes) - don't re-encrypt it
+        if (this.keys.length > 0 && !this.include(key)) {
+          continue
+        }
+
+        if (isEncrypted(value)) { // only re-encrypt those already encrypted
+          row.keys.push(key) // track key(s)
+
+          const decryptedValue = decryptKeyValue(key, value, privateKeyName, existingPrivateKey) // get decrypted value
+
+          const encryptedValue = encryptValue(decryptedValue, newPublicKey) // encrypt with the new publicKey
+
+          envSrc = replace(envSrc, key, encryptedValue)
+        }
+      }
+      row.envSrc = envSrc
+
+      // .env.keys - TODO: for dotenvx pro .env.keys file does not exist
+      row.privateKeyAdded = true
+      row.privateKeyName = privateKeyName
+      row.privateKey = newPrivateKey
+      envKeysSrc = append(envKeysSrc, privateKeyName, newPrivateKey) // append privateKey
+      this.envKeysSources[envKeysFilepath] = envKeysSrc
+      row.envKeysSrc = envKeysSrc
+
+      this.changedFilepaths.add(envFilepath)
+    } catch (e) {
+      if (e.code === 'ENOENT') {
+        row.error = new Errors({ envFilepath, filepath }).missingEnvFile()
+      } else {
+        row.error = e
+      }
+    }
+
+    this.processedEnvs.push(row)
+  }
+
+  _keys () {
+    if (!Array.isArray(this.key)) {
+      return [this.key]
+    }
+
+    return this.key
+  }
+
+  _excludeKeys () {
+    if (!Array.isArray(this.excludeKey)) {
+      return [this.excludeKey]
+    }
+
+    return this.excludeKey
+  }
+
+  _detectEncoding (filepath) {
+    return detectEncoding(filepath)
+  }
+}
+
+module.exports = Rotate