@shieldfive/cli 0.3.1 → 0.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -5,6 +5,78 @@ All notable changes to `@shieldfive/cli` will be documented in this file.
5
5
  The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
6
6
  and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
7
7
 
8
+ ## Unreleased
9
+
10
+ ## 0.5.0 — 2026-11-16
11
+
12
+ ### Behaviour changes
13
+
14
+ - **`sf push` copies the whole folder tree.** It creates one vault folder named
15
+ after the source and recreates every subfolder inside it (folder keys and
16
+ names made as the web app's Upload folder makes them). It used to upload only
17
+ the files at the top of the folder, to the vault root. Through the agent it
18
+ still does that, and refuses a folder with subfolders.
19
+ - **`sf push` is resumable.** A 0600 state file in `~/.shieldfive/push/` records
20
+ finished folders and files; rerunning skips them. A file whose finalize was
21
+ cut off is settled by asking the server, not uploaded twice.
22
+
23
+ ### Added
24
+
25
+ - Retries with backoff for network errors and 5xx/408/429 answers on the
26
+ session, the direct PUT, each part and the folder calls; 3 files at a time
27
+ (`SF_PUSH_CONCURRENCY`); per-file output and a progress line every 10 s;
28
+ first Ctrl-C finishes uploads in progress, second quits.
29
+ - The access token is refreshed during a long push, including in the middle of
30
+ a large file.
31
+ - File and folder names are sent in Unicode NFC.
32
+
33
+ ### Verified
34
+
35
+ - Run against production on 2026-11-16 with the canary account: 248 files /
36
+ 534 MB in 10 folders (520 MB and 12 MB files, Unicode/long/duplicate names),
37
+ killed and interrupted midway, resumed; all 247 non-empty files `ready` with
38
+ the recorded ids, sizes and folders; three of them (including the 520 MB
39
+ file) downloaded byte-identical in the web app; the 0-byte file skipped
40
+ (production does not accept empty files yet).
41
+
42
+ ## 0.4.0 — 2026-10-12
43
+
44
+ ### Behaviour changes
45
+
46
+ - `sf push` now skips dotfiles (`.env`, `.git`, ...) by default; pass
47
+ `--include-dotfiles` to upload them as before.
48
+ - Sign-in refuses a server answer of auth version 1 for an account previously
49
+ signed in with version 2; pass `--allow-legacy-login` to override.
50
+ - Symlinks are skipped instead of followed.
51
+
52
+ ### Security
53
+
54
+ - **Symlinks are not followed.** `push`, `sync` and the agent list files with
55
+ `lstat` and skip symlinks, and files are opened with `O_NOFOLLOW`, so a link
56
+ inside the folder can no longer upload a file from outside it.
57
+ - **`push` skips dotfiles by default** (`.env`, `.git`, ...), as `sync` already
58
+ did. `sf push --include-dotfiles` restores the old behaviour.
59
+ - **Sync manifest written 0600.** `.shieldfive-sync.json` lists file names and
60
+ vault ids; it is no longer world-readable.
61
+ - **README and npm description** now state the scheme precisely: contents are
62
+ AES-256-GCM under a post-quantum hybrid key, names are AES-256-GCM under the
63
+ vault key, and size, part hashes and timestamps are visible to the server.
64
+
65
+ ### Fixed
66
+
67
+ - A failed upload cancels its pending session (`POST /api/files/delete`, as the
68
+ web app does) instead of leaving a row holding quota until the reaper runs.
69
+ Not done when only the finalize answer was lost, since the file may be in.
70
+ - 0-byte files: uploaded as a header-only cipher-v3 object with the server's
71
+ empty-file proof when `/api/files/upload-capabilities` says `emptyCipherV3`;
72
+ otherwise skipped with a clear message (exit code 2) before any session.
73
+ - New uploads send a v6 row-bound encrypted name from upload start (#14).
74
+ - **Downgrade pin for sign-in.** After a sign-in with a login secret (auth
75
+ version 2), the CLI records the account (a SHA-256 of the email) in
76
+ `~/.shieldfive/auth-pins.json` (mode 0600). A later server answer of
77
+ version 1 for that account is refused before the raw password is sent.
78
+ `--allow-legacy-login` on `sf login`, `sf push` or `sf sync` overrides it.
79
+
8
80
  ## 0.3.1 - 2026-10-01
9
81
 
10
82
  ### Security
package/README.md CHANGED
@@ -1,24 +1,35 @@
1
1
  # @shieldfive/cli
2
2
 
3
3
  > Command-line client for [ShieldFive](https://shieldfive.com). Encrypt a folder
4
- > on your machine, then push or continuously sync it to your EU-hosted,
5
- > post-quantum vault. Everything is encrypted locally before it leaves your
6
- > machine. Read the code and verify that for yourself.
4
+ > on your machine, then push or continuously sync it to your EU-hosted vault.
5
+ > File contents and names are encrypted locally before they leave your machine
6
+ > (contents with a post-quantum hybrid key). Read the code and verify that for
7
+ > yourself.
7
8
 
8
9
  [![CI](https://github.com/shieldfive/cli/actions/workflows/ci.yml/badge.svg)](https://github.com/shieldfive/cli/actions/workflows/ci.yml)
9
10
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](LICENSE)
10
11
  [![Status](https://img.shields.io/badge/status-early-yellow.svg)](#honest-scope)
11
12
 
12
13
  The point of this tool being open source is that you do not have to trust a
13
- marketing claim. The plaintext of your files is never sent to a ShieldFive
14
- server. The only bytes that leave your machine are ciphertext (post-quantum
15
- hybrid: ML-KEM-1024 + X25519 key encapsulation, AES-256-GCM chunks) and a short
16
- cryptographic proof. Two files carry the whole story:
14
+ marketing claim. The plaintext of your files and their names is never sent to a
15
+ ShieldFive server:
16
+
17
+ - **File contents** — AES-256-GCM chunks under a per-file key that combines a
18
+ random envelope key (wrapped by your vault root key) with an ML-KEM-1024 +
19
+ X25519 hybrid encapsulation to your vault identity.
20
+ - **File names** — AES-256-GCM under a key derived from your vault root key
21
+ (not the post-quantum layer), plus a keyed HMAC of the lowercased name so the
22
+ server can spot duplicates without reading it.
23
+
24
+ What the server does see, by design: the plaintext size of each file, part
25
+ hashes (SHA-1 of ciphertext), the wrapped keys, an upload proof, timestamps and
26
+ your account. Sign-in sends a secret derived from your password, never the
27
+ password itself. Two files carry the whole story:
17
28
 
18
29
  - [`src/uploadCrypto.mjs`](src/uploadCrypto.mjs) — the encryption applied to
19
30
  every chunk and to the filename.
20
- - [`src/upload.mjs`](src/upload.mjs) — the only place bytes are sent over the
21
- network. Every request body is ciphertext.
31
+ - [`src/upload.mjs`](src/upload.mjs) — the only place file bytes are sent over
32
+ the network. Every file body it sends is ciphertext.
22
33
 
23
34
  If you want to see encryption happen without an account at all, run
24
35
  `sf encrypt <folder>` — it encrypts each file locally and prints the ciphertext
@@ -28,18 +39,27 @@ size, uploading nothing.
28
39
 
29
40
  - The underlying cryptography ([`@shieldfive/crypto`](https://github.com/shieldfive/crypto))
30
41
  has **not** undergone an external audit yet. Treat this as early software.
31
- - **Neither upload path has been exercised against production since 2026-09-03.**
32
- The direct (≤ 5 MiB) path was validated end to end against the backend as it
33
- stood at 0.1.0, but the server then moved that path to a presigned S3 PUT and
34
- stopped issuing an upload token. 0.2.0 hard-required that token, so it could
35
- not upload any file of 5 MiB or less at all. This release rewrites the direct
36
- path to the presigned PUT. It is covered by the offline mock test in
37
- [`test/upload.flow.test.mjs`](test/upload.flow.test.mjs) and sends byte-for-byte
38
- the request the web app sends, but it has not been run live.
39
- - The **multipart** path (files larger than 5 MiB) is covered by a byte-for-byte
40
- review against the server contract and a mock end-to-end test that decrypts
41
- every part, but has not yet been exercised against production with a real
42
- large file.
42
+ - **`sf push` was run against production on 2026-11-16** (0.5.0, with the
43
+ dedicated canary account) on a tree of 248 files (534 MB) in 10 folders: a
44
+ 520 MB file, a 12 MB file, 240 small files, an empty folder, a 0-byte file,
45
+ Unicode, 200-character and duplicate names, two `.DS_Store` files and a
46
+ symlink. The first run was killed (SIGKILL) after 81 files, the second was
47
+ stopped with Ctrl-C after 60 more, the third resumed and finished, and a
48
+ fourth uploaded nothing. In the production database all 247 non-empty files
49
+ are `ready`, each with its id, plaintext size and folder exactly as the client
50
+ recorded them, and the 10 folders form the source tree. The hidden files and
51
+ the symlink were skipped. Three of the files (a small one at the top, the
52
+ Unicode-named one and the 520 MB one) were then downloaded in the web app and
53
+ decrypted byte-identical. Uploading left the account's download allowance
54
+ counter unchanged. One `sf sync` pass of a flat folder uploaded both of its
55
+ files to the vault root. Production does not accept 0-byte files yet:
56
+ `sf push` skips them, says so, and exits 2.
57
+ - These `sf push` behaviours are covered by offline tests only and have **not**
58
+ been exercised against production: retries after network errors and 5xx/429
59
+ answers, stopping on a full vault, refreshing the session during a run longer
60
+ than the access token's lifetime, settling a file whose finalize was cut off,
61
+ a file changing while it uploads, a vault folder deleted between runs, and a
62
+ vault password different from the account password.
43
63
  - `sf sync` is **append-only**: it uploads new and changed files; it does not
44
64
  yet mirror local deletions or renames into the vault.
45
65
  - **`sf login` and the agent have not been run against production.** They are
@@ -72,7 +92,7 @@ trust it):
72
92
  git clone https://github.com/shieldfive/cli.git
73
93
  cd cli
74
94
  npm install # @shieldfive/crypto (+ libsodium), @supabase/supabase-js, @noble/*
75
- npm test # 107 tests
95
+ npm test
76
96
  ```
77
97
 
78
98
  ## Commands
@@ -81,7 +101,7 @@ npm test # 107 tests
81
101
  sf login [--idle=<hours>] sign in once; an agent holds the session in memory (locks after 8 h idle)
82
102
  sf status is an agent running, and for which account
83
103
  sf logout [--everywhere] revoke the session and stop the agent; --everywhere signs out every device
84
- sf push <folder> encrypt and upload every file once
104
+ sf push <folder> [--include-dotfiles] copy a whole folder tree into a new vault folder; rerun to resume
85
105
  sf sync <folder> [--watch] [--interval=N] upload new/changed files; --watch keeps a poll loop (N seconds, default 5)
86
106
  sf verify <file>... report whether each file is safely stored in your vault
87
107
  sf encrypt <folder> encrypt each file locally, upload nothing (no account needed)
@@ -89,7 +109,36 @@ sf inspect <folder> read-only recursive migration inven
89
109
  ```
90
110
 
91
111
  `sf push` and `sf sync` pick the direct or multipart upload path automatically
92
- from the file size.
112
+ from the file size. Symlinks are never followed (files are opened with
113
+ `O_NOFOLLOW`), and names starting with `.` (`.DS_Store`, `.git`, `.env`) are
114
+ skipped unless you pass `sf push --include-dotfiles`. An empty (0-byte) file is
115
+ uploaded only when the server says it accepts empty files; otherwise it is
116
+ reported as skipped and the exit code is 2. If an upload fails after its session
117
+ was created, the pending session is cancelled so it does not hold quota.
118
+
119
+ `sf push` copies the whole tree: it creates one vault folder named after
120
+ `<folder>` and recreates every subfolder (empty ones too) inside it, the way the
121
+ web app's Upload folder does. Three files upload at a time
122
+ (`SF_PUSH_CONCURRENCY`, 1-8); large files also send three parts at a time
123
+ (`SF_UPLOAD_CONCURRENCY`). Network errors, 5xx, 408 and 429 answers are retried
124
+ with backoff. It prints each file as it finishes and a progress line every 10
125
+ seconds. A full vault or a refused session stops the run; any other file that
126
+ fails is listed, the rest continue, and the exit code is 2.
127
+
128
+ It is resumable. A state file in `~/.shieldfive/push/` (mode 0600; file names
129
+ and vault ids, never keys) records each folder made and each file once its
130
+ upload is final, so running the same command again skips those and uploads the
131
+ rest; a file interrupted halfway starts again from its beginning. The first
132
+ Ctrl-C lets the uploads in progress finish and exits 130; a second quits at once.
133
+ A run killed during a file's last step is settled on the next run by asking the
134
+ server whether that file arrived, so it is not uploaded twice. A file that
135
+ changes while it uploads is reported, and uploaded again on the next run. Files
136
+ you delete from the vault after a run are not re-uploaded by a rerun.
137
+
138
+ Through the agent (`sf login`), `sf push` uploads only the files at the top of
139
+ the folder, to the vault root, and refuses a folder with subfolders; use the
140
+ environment variables below for a tree. `sf sync` is also top-level only and
141
+ uploads to the vault root.
93
142
 
94
143
  `sf inspect` reports JSON counts of files, directories, bytes and skipped entries.
95
144
  It includes nested/hidden files and empty folders, reads metadata only, and
@@ -164,6 +213,32 @@ With `sf login` you are asked once. The agent refreshes the stepped-up session
164
213
  on its own, so `sf sync --watch` keeps working on a two-factor account until the
165
214
  agent locks or you log out.
166
215
 
216
+ ## Migrating from Dropbox or Google Drive with rclone
217
+
218
+ For most people the web app is simpler: sync Dropbox to your computer, then
219
+ **Upload > Upload folder** in the browser
220
+ (see [Move from Dropbox to ShieldFive](https://shieldfive.com/help/move-from-dropbox)).
221
+ For a large or scripted move:
222
+
223
+ ```sh
224
+ # 1. Copy your cloud files to a local folder (rclone.org; `rclone config` first)
225
+ rclone copy dropbox: ~/dropbox-export --progress
226
+ # Google Drive: rclone copy gdrive: ~/drive-export --progress
227
+
228
+ # 2. Encrypt and upload it, keeping the folder structure
229
+ export SF_EMAIL=you@example.com
230
+ read -rs SF_PASSWORD; echo; export SF_PASSWORD
231
+ sf push ~/dropbox-export
232
+
233
+ # 3. If it stops for any reason, run the same command again: it resumes.
234
+ sf push ~/dropbox-export
235
+ ```
236
+
237
+ Your vault gets a folder named `dropbox-export` with the same tree inside. The
238
+ command exits 0 only when every file is in the vault. Uploads do not count
239
+ toward your monthly download allowance. `sf push` copies once; it does not keep
240
+ the folder in sync afterwards.
241
+
167
242
  ## What the agent can and cannot do
168
243
 
169
244
  The agent keeps two things in memory: your signed-in session and your unlocked
package/package.json CHANGED
@@ -1,8 +1,8 @@
1
1
  {
2
2
  "name": "@shieldfive/cli",
3
- "version": "0.3.1",
3
+ "version": "0.5.0",
4
4
  "type": "module",
5
- "description": "Open-source command-line client for ShieldFive: encrypt a folder on your machine and sync it to your EU-hosted vault with post-quantum hybrid encryption (ML-KEM-1024 + X25519). Read the code; it encrypts before upload.",
5
+ "description": "Open-source command-line client for ShieldFive: encrypts files on your machine before upload (contents: AES-256-GCM under an ML-KEM-1024 + X25519 hybrid key; names: AES-256-GCM under your vault key) and pushes or syncs them to your vault. Read the code.",
6
6
  "license": "Apache-2.0",
7
7
  "author": "Cho Garcia <security@shieldfive.com>",
8
8
  "homepage": "https://github.com/shieldfive/cli",
@@ -5,7 +5,7 @@
5
5
  // See docs/agent-design.md for why it exists and what it will never do.
6
6
 
7
7
  import { createHmac, randomUUID } from 'node:crypto'
8
- import { chmod, open, readFile, rm, stat } from 'node:fs/promises'
8
+ import { chmod, lstat, open, readFile, rm, stat } from 'node:fs/promises'
9
9
  import { connect, createServer } from 'node:net'
10
10
  import { basename, dirname, join } from 'node:path'
11
11
 
@@ -189,7 +189,9 @@ export function createAgent({
189
189
  }
190
190
 
191
191
  async function upload(path) {
192
- const before = await stat(path)
192
+ // lstat: a symlink is refused, not followed (and readFileChunks opens with
193
+ // O_NOFOLLOW, so a swap after this check fails too).
194
+ const before = await lstat(path)
193
195
  if (!before.isFile()) throw new AgentError('not_a_file', `${path} is not a regular file.`)
194
196
  const token = await accessToken()
195
197
 
package/src/auth.mjs CHANGED
@@ -87,6 +87,10 @@ export async function signIn({
87
87
  getTotpCode,
88
88
  fetchImpl = fetch,
89
89
  createClientImpl = createClient,
90
+ // { isPinned(email), pin(email) } -- src/authPin.mjs fileAuthPin(). Null
91
+ // means no pin (tests, embedders); the sf commands always pass one.
92
+ authPin = null,
93
+ allowLegacyLogin = false,
90
94
  }) {
91
95
  // The raw password is never handed to Supabase directly. The salt lookup
92
96
  // decides what the version-aware login route receives: only the derived
@@ -94,7 +98,10 @@ export async function signIn({
94
98
  // server affirmatively reported as version 1. test/auth.noPasswordLeak.test.mjs
95
99
  // pins this across every path.
96
100
  email = String(email ?? '').trim().toLowerCase()
97
- const credentials = await buildSignInCredentials({ apiBaseUrl, email, password, fetchImpl })
101
+ const pinnedV2 = authPin ? await authPin.isPinned(email) : false
102
+ const credentials = await buildSignInCredentials({
103
+ apiBaseUrl, email, password, fetchImpl, pinnedV2, allowLegacyLogin,
104
+ })
98
105
  let response
99
106
  try {
100
107
  response = await fetchImpl(new URL('/api/mobile/auth/login', apiBaseUrl), {
@@ -127,6 +134,13 @@ export async function signIn({
127
134
  throw new Error('Sign-in returned no access token.')
128
135
  }
129
136
 
137
+ // Signed in with only the login secret: the account is version 2. Remember
138
+ // it so a later version-1 answer is refused (src/authPin.mjs). A failed
139
+ // write does not fail the sign-in; the next one retries it.
140
+ if (authPin && !('password' in credentials)) {
141
+ try { await authPin.pin(email) } catch { /* retried on the next sign-in */ }
142
+ }
143
+
130
144
  const aal2Token = await stepUpMfaIfRequired({ supabase, getTotpCode })
131
145
 
132
146
  // The refresh token has to come from the session that was stepped up.
@@ -0,0 +1,95 @@
1
+ // "This account signs in with a login secret" -- remembered on this computer.
2
+ //
3
+ // /api/auth/salt is the server telling the CLI how to sign in: version 2 means
4
+ // send only the derived login secret, version 1 means send the raw password.
5
+ // A server that has been tampered with (or an account a stolen session
6
+ // downgraded) can answer version 1 for an account that is really version 2,
7
+ // and a CLI that obeys sends the raw password -- the one thing version 2
8
+ // promises never leaves this machine. The mobile app and the web already pin;
9
+ // this is the CLI's pin.
10
+ //
11
+ // Once a sign-in has gone through with the server saying version 2, the
12
+ // account is recorded here. Afterwards a version-1 answer for it is refused
13
+ // unless the user passes --allow-legacy-login.
14
+ //
15
+ // File: ~/.shieldfive/auth-pins.json, mode 0600 in a 0700 directory. It holds
16
+ // SHA-256 hashes of lower-cased email addresses, not the addresses, so the file
17
+ // does not list which accounts this computer uses.
18
+
19
+ import { createHash } from 'node:crypto'
20
+ import { chmod, readFile, rename, writeFile } from 'node:fs/promises'
21
+ import { homedir } from 'node:os'
22
+ import { join } from 'node:path'
23
+
24
+ import { ensurePrivateDir, shieldfiveHome } from './agent/paths.mjs'
25
+
26
+ export const AUTH_PIN_FILE = 'auth-pins.json'
27
+
28
+ export class LegacyLoginRefusedError extends Error {
29
+ constructor() {
30
+ super(
31
+ 'Refusing to sign in: this computer has signed in to this account with a ' +
32
+ 'login secret before, but the server now asks for the raw password ' +
33
+ '(auth version 1). That should not happen to an account that has moved ' +
34
+ 'to version 2, so the password was not sent. If you know why (for ' +
35
+ 'example ShieldFive support reset the account), run the command again ' +
36
+ 'with --allow-legacy-login.',
37
+ )
38
+ this.name = 'LegacyLoginRefusedError'
39
+ this.code = 'legacy_login_refused'
40
+ }
41
+ }
42
+
43
+ function emailKey(email) {
44
+ return createHash('sha256')
45
+ .update(String(email ?? '').trim().toLowerCase())
46
+ .digest('hex')
47
+ }
48
+
49
+ /**
50
+ * The pin store backed by ~/.shieldfive/auth-pins.json. `home` is injectable
51
+ * for tests. A file that cannot be read as a pin list is treated as holding
52
+ * no pins for reads, and is replaced on the next write.
53
+ */
54
+ export function fileAuthPin({ home = homedir() } = {}) {
55
+ const dir = shieldfiveHome(home)
56
+ const file = join(dir, AUTH_PIN_FILE)
57
+
58
+ const readKeys = async () => {
59
+ let raw
60
+ try {
61
+ raw = await readFile(file, 'utf8')
62
+ } catch (err) {
63
+ if (err.code === 'ENOENT') return []
64
+ throw err
65
+ }
66
+ try {
67
+ const parsed = JSON.parse(raw)
68
+ return Array.isArray(parsed?.v2)
69
+ ? parsed.v2.filter((k) => typeof k === 'string' && /^[0-9a-f]{64}$/.test(k))
70
+ : []
71
+ } catch {
72
+ return []
73
+ }
74
+ }
75
+
76
+ return {
77
+ file,
78
+ async isPinned(email) {
79
+ return (await readKeys()).includes(emailKey(email))
80
+ },
81
+ async pin(email) {
82
+ await ensurePrivateDir(dir)
83
+ const keys = await readKeys()
84
+ const key = emailKey(email)
85
+ if (keys.includes(key)) return
86
+ keys.push(key)
87
+ // Write-then-rename so a crash never leaves a half-written file, and
88
+ // chmod after rename in case an older file had looser bits.
89
+ const tmp = `${file}.${process.pid}.tmp`
90
+ await writeFile(tmp, JSON.stringify({ v2: keys }) + '\n', { mode: 0o600 })
91
+ await rename(tmp, file)
92
+ await chmod(file, 0o600)
93
+ },
94
+ }
95
+ }