@sharedenv/cli 0.1.0 → 1.1.0

package/README.md

@@ -3,7 +3,7 @@
 **The zero-knowledge secrets vault for teams, in your terminal.**
 
 [![npm](https://img.shields.io/npm/v/@sharedenv/cli)](https://www.npmjs.com/package/@sharedenv/cli)
-[![license](https://img.shields.io/npm/l/@sharedenv/cli)](LICENSE)
+[![license](https://img.shields.io/github/license/SharedEnv/cli)](LICENSE)
 
 Your `.env` files have no business in Slack, in an email or in a git repo. With
 [sharedenv](https://sharedenv.com), your team keeps its environment variables, API keys and
@@ -64,15 +64,112 @@ is passed through, so scripts and CI behave exactly as without sharedenv.
 A variable your shell already sets wins over the vault, so a local `DATABASE_URL` is never
 replaced behind your back: add `--override-env` when you want the vault's value.
 
-### Write a .env file
+### Get your secrets as a file, or as JSON
 
 ```bash
-sharedenv pull prod --out .env       # a file only you can read
-sharedenv pull prod --format json    # for scripts and tools
+sharedenv pull prod                    # dotenv, on stdout
+sharedenv pull prod --out .env         # the same, in a file only you can read
+sharedenv pull prod --format json      # JSON, for scripts and programs
 ```
 
-The file works with dotenv-style libraries and can be sourced by a shell
-(`set -a; . ./.env; set +a`): every value is quoted so that nothing in it gets executed.
+stdout carries the secrets and nothing else (questions and messages go to stderr), so
+pipes and redirections get clean data. From a script or a program there is no terminal to
+ask for your master password: set `SHAREDENV_PASSWORD`.
+
+#### dotenv (the default)
+
+For a vault holding five secrets, `sharedenv pull prod` prints:
+
+```dotenv
+APP_NAME='Acme billing'
+DATABASE_URL=postgres://app:s3cret@db.internal:5432/app
+JWT_PRIVATE_KEY='-----BEGIN PRIVATE KEY-----
+MIIEvQIBADANBgkqhkiG9w0BAQEFAASC
+-----END PRIVATE KEY-----'
+SMTP_PASSWORD="it's a pass"
+STRIPE_KEY=sk_test_example
+```
+
+- One `NAME=value` per secret, sorted by name, so two pulls of the same vault give the
+  same file.
+- A value is written as is when it holds only plain characters, between single quotes
+  otherwise (multi-line values such as a PEM key included), and between double quotes when
+  it contains a single quote.
+- Nothing is ever expanded or executed when the file is read. A value that no quoting can
+  carry safely (a single quote together with `"`, `$`, a backtick or `\`) is refused with
+  its name: use `--format json` for that vault.
+
+Every common reader loads it as is:
+
+```bash
+set -a; . ./.env; set +a                 # a shell (bash, zsh, sh)
+node --env-file=.env server.js           # Node.js 20.12 or later
+```
+
+```js
+import 'dotenv/config'; // Node.js, with the dotenv package
+```
+
+```python
+from dotenv import load_dotenv  # Python, with python-dotenv
+load_dotenv()
+```
+
+One exception: `docker run --env-file` reads values literally (it would keep the quotes)
+and refuses multi-line values. Hand the variables to Docker through `run` instead:
+
+```bash
+sharedenv run prod -- docker run -e DATABASE_URL -e STRIPE_KEY my-app
+```
+
+#### JSON
+
+`sharedenv pull prod --format json` prints one object, names sorted, every value a string:
+
+```json
+{
+  "APP_NAME": "Acme billing",
+  "DATABASE_URL": "postgres://app:s3cret@db.internal:5432/app",
+  "JWT_PRIVATE_KEY": "-----BEGIN PRIVATE KEY-----\nMIIEvQIBADANBgkqhkiG9w0BAQEFAASC\n-----END PRIVATE KEY-----",
+  "SMTP_PASSWORD": "it's a pass",
+  "STRIPE_KEY": "sk_test_example"
+}
+```
+
+One value, from a shell:
+
+```bash
+sharedenv pull prod --format json | jq -r .DATABASE_URL
+```
+
+From a program:
+
+```js
+// Node.js
+import { execFileSync } from 'node:child_process';
+
+const secrets = JSON.parse(
+  execFileSync('sharedenv', ['pull', 'prod', '--format', 'json'], { encoding: 'utf8' })
+);
+console.log(secrets.DATABASE_URL);
+```
+
+```python
+# Python
+import json
+import subprocess
+
+secrets = json.loads(
+    subprocess.run(
+        ["sharedenv", "pull", "prod", "--format", "json"],
+        check=True, capture_output=True, text=True,
+    ).stdout
+)
+print(secrets["DATABASE_URL"])
+```
+
+If the command fails (wrong password, unknown vault, no network), it exits with a non-zero
+code and says why on stderr: `execFileSync` throws, and `check=True` raises.
 
 ### Accept what your team shares with you
 
@@ -151,6 +248,11 @@ Good practice for pipelines:
   memory for the command, and never written anywhere.
 - **You know who shares with you.** A share opens only if it was sealed by its sender, and
   the CLI remembers the keys it has seen: if someone's key ever changes, it stops and asks.
+- **A key cannot be swapped in our database without a trace.** Every public key the server
+  hands out is first written to a log that only grows, each entry chained to the previous
+  one by its hash. The CLI trusts a key only if it is the last one logged for its owner,
+  warns you when a key that is not yours is logged under your name, and refuses a log that
+  rewrites what it already checked.
 - **A vault cannot take over your machine.** Anyone who can edit a vault chooses its
   values, so `run` holds back variables that would make a program run code or reroute its
   traffic (`LD_*`, `PATH`, `NODE_OPTIONS`, proxies, custom certificates...) unless you pass
@@ -163,8 +265,10 @@ What it does not protect against, said plainly:
 - **Names.** Vault names and secret names (like `DATABASE_URL`) are stored in clear, so
   that they can be listed and organized; values, and the URL and username that go with
   them, are encrypted. Never put a secret in a name.
-- **The first contact.** The first key the CLI sees for someone is the one our directory
-  serves; comparing fingerprints once settles it for good.
+- **A swap is caught, not prevented.** Someone who can write to our database could still
+  log a key of their own: the log keeps the trace, and its owner is warned the next time
+  they unlock. Someone who controls our servers themselves could show you a different log.
+  Comparing fingerprints once with someone settles it for good.
 - **After you accept a share**, its sender is not checked again; and an older value of the
   same secret could be served back to you.
 - **Your own account on your own machine.** The session file (`~/.sharedenv/session.json`,
@@ -194,9 +298,9 @@ What it does not protect against, said plainly:
 | `SHAREDENV_API`        | API URL (default `https://api.sharedenv.com`).          |
 | `SHAREDENV_HOME`       | Where the CLI keeps its files (default `~/.sharedenv`). |
 
-The CLI keeps two files there, both readable by you only: `session.json` (your session)
-and `pins.json` (the public keys this machine has checked). Never a secret, never your
-password, never your private key.
+The CLI keeps three files there, all readable by you only: `session.json` (your session),
+`pins.json` (the public keys this machine has checked) and `log.json` (how far it checked
+the public key log). Never a secret, never your password, never your private key.
 
 Exit codes: `0` success, `1` error, `2` wrong usage, `130` cancelled. With `run`: the
 command's own code, `128 + n` when signal `n` ended it, `127` when the command does not

package/bin/sharedenv.js

@@ -12,7 +12,8 @@ import { Writable } from 'stream';
 import { parseArgs } from 'util';
 import { fingerprintOf } from '../lib/crypto.js';
 import { apiBase, createClient, expectOk, listAll } from '../lib/api.js';
-import { login, publicKeyOf, unlockPrivateKey } from '../lib/auth.js';
+import { login, publicKeyOf, unlockAccount } from '../lib/auth.js';
+import { syncLog, watchWarnings } from '../lib/transparency.js';
 import { childEnv, splitUnsafeEnv } from '../lib/env.js';
 import { FORMATS } from '../lib/format.js';
 import { runChild } from '../lib/run.js';
@@ -121,10 +122,25 @@ function saveSession() {
   connected.session.cookies = cookies;
 }
 
+// Each time the vault opens: is the key the public log holds for us still ours? A
+// warning, never a refusal - reading your own secrets seals nothing to anyone.
+async function watchOwnKey(client, { privateKey, userId }) {
+  try {
+    const report = await syncLog(client, {
+      store,
+      watch: { [`user:${userId}`]: publicKeyOf(privateKey) },
+    });
+    for (const line of watchWarnings(report)) note(line);
+  } catch (err) {
+    note(`WARNING: ${err.message}.`);
+  }
+}
+
 async function unlock() {
   const client = connect();
-  const privateKey = await unlockPrivateKey(client, await masterPassword());
-  return { client, privateKey };
+  const account = await unlockAccount(client, await masterPassword());
+  await watchOwnKey(client, account);
+  return { client, privateKey: account.privateKey };
 }
 
 // ── Commands ────────────────────────────────────────────────────────────────────────
@@ -148,9 +164,10 @@ async function cmdLogin({ values }) {
   const client = createClient({ baseUrl, cookies, userAgent: USER_AGENT });
   const otp = async () => process.env.SHAREDENV_OTP || (await ask('Two-factor code: '));
   await login(client, { identifier, password, otp });
-  await unlockPrivateKey(client, password); // the vault opens: only then is anything kept
+  const account = await unlockAccount(client, password); // the vault opens: only then is anything kept
   store.writeSession({ baseUrl, cookies: client.cookies() });
   note(`Signed in to ${baseUrl}.`);
+  await watchOwnKey(client, account);
 }
 
 async function cmdPull({ values, positionals: [ref] }) {
@@ -311,9 +328,10 @@ async function cmdAccept({ values, positionals: [ref] }) {
   const { client, privateKey } = await unlock();
   const safeId = await resolvePendingShare(client, ref, values.team);
   const confirm = confirmShare(values);
+  const log = { store };
   const result = values.team
-    ? await acceptTeamShare(client, privateKey, values.team, safeId, { confirm })
-    : await acceptShare(client, privateKey, safeId, { confirm });
+    ? await acceptTeamShare(client, privateKey, values.team, safeId, { confirm, log })
+    : await acceptShare(client, privateKey, safeId, { confirm, log });
   note(result.accepted ? 'Accepted.' : 'Left pending.');
 }
 

package/lib/auth.js

@@ -42,6 +42,11 @@ export function publicKeyOf(privateKey) {
 
 // → the account's private key, in memory only.
 export async function unlockPrivateKey(client, password) {
+  return (await unlockAccount(client, password)).privateKey;
+}
+
+// → { privateKey, userId }: the key, and the name it is logged under (transparency.js).
+export async function unlockAccount(client, password) {
   const keyring = expectOk(await client.call('GET', '/keyring/me'), 'cannot read your keyring');
   if (!keyring?.encPrivateKey) {
     throw new Error('this account has no keyring yet: open the web app once to create it');
@@ -66,5 +71,5 @@ export async function unlockPrivateKey(client, password) {
   if (String(keyring.publicKey || '').toLowerCase() !== publicKeyOf(privateKey)) {
     throw new Error('the public key on file is not the one of your private key — refusing');
   }
-  return privateKey;
+  return { privateKey, userId: keyring.userId };
 }

package/lib/crypto/kt.js

@@ -0,0 +1,100 @@
+// Key transparency — the public-key directory as a public, append-only log.
+//
+// The server is the key directory, and the directory is only rows in a database: whoever
+// can write there (a leak, an insider, a bug) can answer "Bob" with a key of their own at
+// the moment Alice seals to Bob. A client cannot tell a substituted key from a real one.
+// What it CAN check is that every key is on the record, and that the record never
+// changes behind its back:
+//
+//   1. Every key the server hands out is first written to a log of entries that only
+//      grows. Entry n carries the hash of entry n-1, so the head hash sums up the whole
+//      history: rewriting one past entry changes every hash after it.
+//   2. A client seals to a key only if it is the LAST key the log holds for that
+//      subject. A substituted key must therefore be logged, in public.
+//   3. Each owner watches the log for their own name. A key logged under it that is
+//      not theirs is reported: the substitution is caught, and the log is the proof.
+//   4. Each client keeps the log it checked: a past entry rewritten, or a log shorter
+//      than what it already saw, is refused.
+//
+// Entries hold HASHES of the subject and of the key, never the ids or the keys: the
+// log is readable by anyone and only says "some key of some subject changed at entry
+// n". Whoever already knows a subject and a key (the person sealing to them) computes
+// the same hashes. Subjects: `user:<id>`, `team:<id>`.
+//
+// It only does arithmetic on hashes. Every client (browser, extension, command line)
+// computes the same hashes for the same entries.
+
+import crypto from 'crypto';
+
+export const KT_GENESIS = '0'.repeat(64);
+
+const sha256 = (text) => crypto.createHash('sha256').update(text, 'utf8').digest('hex');
+
+export const subjectHash = (subject) => sha256(`sharedenv/kt/subject/v1|${subject}`);
+export const keyHash = (publicKeyHex) =>
+  sha256(`sharedenv/kt/key/v1|${String(publicKeyHex).toLowerCase()}`);
+export const entryHash = ({ seq, prev, subject, key }) =>
+  sha256(`sharedenv/kt/entry/v1|${seq}|${prev}|${subject}|${key}`);
+
+// The next entry after `head` ({ seq, hash }), for a subject and a key.
+export function nextEntry(head, subject, publicKeyHex) {
+  const entry = {
+    seq: head.seq + 1,
+    prev: head.hash,
+    subject: subjectHash(subject),
+    key: keyHash(publicKeyHex),
+  };
+  return { ...entry, hash: entryHash(entry) };
+}
+
+export class KtError extends Error {
+  constructor(code, message) {
+    super(message);
+    this.name = 'KtError';
+    this.code = code;
+  }
+}
+
+// What a client knows of the log: its head and the last key of every subject.
+export const emptyView = () => ({ seq: 0, hash: KT_GENESIS, latest: {} });
+
+// Replays `entries` (ascending, starting right after `view.seq`) onto a copy of `view`.
+//   watch    { [subject]: publicKeyHex } — subjects whose key WE know (our own, a team
+//            whose private key we hold). Once their key is in the log, any later entry
+//            for them with another key is reported.
+// -> { view, foreign, unlogged }
+//   foreign    [{ subject, seq }] — a key that is not ours, logged under our name
+//   unlogged   [subject] — our key is not the last one the log holds for us
+// Throws KtError BROKEN: a gap, or a hash that does not chain.
+export function replay(view, entries, { watch = {} } = {}) {
+  const next = { seq: view.seq, hash: view.hash, latest: { ...view.latest } };
+  const watched = Object.entries(watch).map(([subject, publicKey]) => ({
+    subject,
+    hashed: subjectHash(subject),
+    key: keyHash(publicKey),
+    armed: false,
+  }));
+  for (const w of watched) w.armed = next.latest[w.hashed] === w.key;
+
+  const foreign = [];
+  for (const e of entries) {
+    if (e.seq !== next.seq + 1 || e.prev !== next.hash || entryHash(e) !== e.hash) {
+      throw new KtError('BROKEN', `the log does not chain at entry ${e.seq}`);
+    }
+    next.seq = e.seq;
+    next.hash = e.hash;
+    next.latest[e.subject] = e.key;
+    for (const w of watched) {
+      if (e.subject !== w.hashed) continue;
+      if (e.key === w.key) w.armed = true;
+      else if (w.armed) foreign.push({ subject: w.subject, seq: e.seq });
+    }
+  }
+
+  const unlogged = watched.filter((w) => next.latest[w.hashed] !== w.key).map((w) => w.subject);
+  return { view: next, foreign, unlogged };
+}
+
+// Is `publicKeyHex` the last key the log holds for `subject`?
+export const isLogged = (view, subject, publicKeyHex) =>
+  view.latest[subjectHash(subject)] === keyHash(publicKeyHex);

package/lib/crypto.js

@@ -8,3 +8,4 @@ export {
 } from './crypto/kdf.js';
 export { openSecret } from './crypto/secretBlob.js';
 export { fingerprintOf } from './crypto/fingerprint.js';
+export { emptyView, replay, isLogged } from './crypto/kt.js';

package/lib/shares.js

@@ -5,6 +5,7 @@
 
 import { Wallet, fingerprintOf } from './crypto.js';
 import { expectOk, listAll } from './api.js';
+import { assertLogged, syncLog } from './transparency.js';
 
 const wallet = new Wallet();
 
@@ -73,10 +74,27 @@ export async function openPendingShare(client, privateKey, safeId) {
   return { share, senderKey, fingerprint: fingerprintOf(senderKey) };
 }
 
+// The sender's key (and the team key a team share moves to) is the directory's word:
+// it must be the last key the public key log holds for them. `log`: { store }.
+async function assertSenderLogged(client, log, opened, team) {
+  const { view } = await syncLog(client, log);
+  assertLogged(view, [
+    {
+      subject: `user:${opened.share.from?._id}`,
+      publicKey: opened.senderKey,
+      label: senderLabel(opened.share),
+    },
+    ...(team
+      ? [{ subject: `team:${team}`, publicKey: opened.teamPublicKey, label: 'the team' }]
+      : []),
+  ]);
+}
+
 // Check, then accept. `confirm(opened)` (async → boolean) is where a human compares the
 // fingerprint; a no leaves the share pending.
-export async function acceptShare(client, privateKey, safeId, { confirm } = {}) {
+export async function acceptShare(client, privateKey, safeId, { confirm, log } = {}) {
   const opened = await openPendingShare(client, privateKey, safeId);
+  await assertSenderLogged(client, log, opened);
   if (confirm && !(await confirm(opened))) return { ...opened, accepted: false };
   expectOk(await client.call('POST', `/shares/${safeId}/accept`), 'cannot accept the share');
   return { ...opened, accepted: true };
@@ -140,8 +158,9 @@ export async function openPendingTeamShare(client, privateKey, teamId, safeId) {
 
 // Check, re-seal to the team's current key (by us), accept. The grant version we opened
 // travels with it: a rotation in between makes it a 409, never an old DEK.
-export async function acceptTeamShare(client, privateKey, teamId, safeId, { confirm } = {}) {
+export async function acceptTeamShare(client, privateKey, teamId, safeId, { confirm, log } = {}) {
   const opened = await openPendingTeamShare(client, privateKey, teamId, safeId);
+  await assertSenderLogged(client, log, opened, teamId);
   // The DEK never moves to a team key nobody vouched for: `confirm` must check it.
   if (!opened.teamKeyVouched && !confirm) {
     throw new Error(

package/lib/store.js

@@ -1,6 +1,7 @@
 // What the CLI keeps on disk, under ~/.sharedenv (or $SHAREDENV_HOME): the session
-// cookies, and the public keys this machine has vouched for (trust.js). Never the
-// password, never a private key, never a secret.
+// cookies, the public keys this machine has vouched for (trust.js), and how far it
+// checked the public key log (transparency.js). Never the password, never a private
+// key, never a secret.
 
 import { randomBytes } from 'crypto';
 import {
@@ -48,6 +49,7 @@ function readJson(file) {
 export function createStore(dir = process.env.SHAREDENV_HOME || join(homedir(), '.sharedenv')) {
   const sessionFile = join(dir, 'session.json');
   const pinsFile = join(dir, 'pins.json');
+  const logFile = join(dir, 'log.json');
   const save = (file, value) => {
     mkdirSync(dir, { recursive: true, mode: 0o700 });
     writePrivate(file, `${JSON.stringify(value, null, 2)}\n`);
@@ -62,5 +64,8 @@ export function createStore(dir = process.env.SHAREDENV_HOME || join(homedir(),
     // → { [subject]: publicKey }
     readPins: () => readJson(pinsFile) || {},
     writePins: (pins) => save(pinsFile, pins),
+    // → { seq, hash, latest } | null — the public key log, as far as it was checked
+    readLogView: () => readJson(logFile),
+    writeLogView: (view) => save(logFile, view),
   };
 }

package/lib/transparency.js

@@ -0,0 +1,99 @@
+// The public key log (key transparency). The server is the key directory, and the
+// directory is rows in a database: whoever can write there could answer "Bob" with a key
+// of their own. So every key it hands out is first written to a log that only grows,
+// each entry chained to the previous one by its hash. This CLI:
+//   - seals to, or trusts the seal of, a key only if it is the LAST one the log holds
+//     for its owner: a substituted key would have to be logged, where its owner sees it ;
+//   - watches your own name: a key logged under it that is not yours is reported ;
+//   - keeps what it checked (~/.sharedenv/log.json) and refuses a log that does not
+//     chain, rewrites what it already saw, or goes back.
+// The arithmetic is lib/crypto/kt.js.
+
+import { emptyView, replay, isLogged } from './crypto.js';
+import { expectOk } from './api.js';
+
+const PAGE_LIMIT = 500;
+
+const saneView = (v) =>
+  v &&
+  Number.isInteger(v.seq) &&
+  typeof v.hash === 'string' &&
+  v.latest &&
+  typeof v.latest === 'object';
+
+// Reads the log after what this machine checked, replays it, keeps the result.
+//   store   lib/store.js (or none: replay from the start, keep nothing)
+//   watch   { [subject]: publicKey } — the keys we hold
+// → { view, foreign, unlogged }. Throws when the log cannot be read or does not add up.
+export async function syncLog(client, { store, watch = {} } = {}) {
+  const stored = store?.readLogView();
+  const from = saneView(stored) ? stored : emptyView();
+
+  const read = async (to, page) => {
+    const bound = to === null ? '' : `&to=${to}`;
+    return expectOk(
+      await client.call(
+        'GET',
+        `/transparency/log?from=${from.seq}${bound}&page=${page}&limit=${PAGE_LIMIT}`
+      ),
+      'cannot read the public key log'
+    );
+  };
+  const first = await read(null, 1);
+  const { head } = first;
+  if (!Number.isInteger(head?.seq) || typeof head?.hash !== 'string') {
+    throw new Error('the public key log sent no head');
+  }
+  if (head.seq < from.seq) {
+    throw new Error('the public key log is shorter than what this machine already checked');
+  }
+  const entries = [...(first.data || [])];
+  for (let page = 2; page <= (first.pagination?.pages ?? 1); page += 1) {
+    entries.push(...((await read(head.seq, page)).data || []));
+  }
+
+  let replayed;
+  try {
+    replayed = replay(from, entries, { watch });
+  } catch (err) {
+    throw new Error(`the public key log does not add up: ${err.message}`);
+  }
+  const { view, foreign, unlogged } = replayed;
+  if (view.seq !== head.seq || view.hash !== head.hash) {
+    throw new Error('the public key log does not add up: its head does not match its entries');
+  }
+  if (store && view.seq > from.seq) store.writeLogView(view);
+  return { view, foreign, unlogged };
+}
+
+// Every target's key must be the last one the log holds for its subject.
+// targets: [{ subject, publicKey, label }]
+export function assertLogged(view, targets) {
+  const missing = targets.filter((t) => !isLogged(view, t.subject, t.publicKey));
+  if (missing.length) {
+    throw new Error(
+      `the server handed out a key the public key log does not hold (${missing
+        .map((t) => t.label || t.subject)
+        .join(', ')}): refusing`
+    );
+  }
+}
+
+// What the watch found, as lines for stderr (none when all is well).
+export function watchWarnings({ foreign = [], unlogged = [] }) {
+  const lines = [];
+  if (foreign.length) {
+    lines.push(
+      'WARNING: a key that is not yours was registered under your name in the public key log ' +
+        `(entry ${foreign.map((f) => f.seq).join(', ')}). If you did not reset your vault, ` +
+        'tell SharedEnv: it is the trace of an attack.'
+    );
+  }
+  if (unlogged.length) {
+    lines.push(
+      'WARNING: your key is not the last one the public key log holds for you: nobody can ' +
+        'share with you until it is. Tell SharedEnv.'
+    );
+  }
+  return lines;
+}

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@sharedenv/cli",
-  "version": "0.1.0",
+  "version": "1.1.0",
   "type": "module",
   "description": "sharedenv CLI — pull and run with your zero-knowledge secrets, decrypted locally.",
   "license": "Apache-2.0",