@shieldfive/cli 0.3.0 → 0.3.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -5,7 +5,51 @@ All notable changes to `@shieldfive/cli` will be documented in this file.
5
5
  The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
6
6
  and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
7
7
 
8
- ## Unreleased
8
+ ## 0.3.1 - 2026-10-01
9
+
10
+ ### Security
11
+
12
+ - **`sf push` and `sf sync` write the web's default format.** They wrote the
13
+ deprecated v0 format (AES-GCM chunks, no header), which the format spec
14
+ forbids for new writes: a server could drop trailing chunks and the file
15
+ still decrypted, could move a file's bytes onto another row, and there was no
16
+ post-quantum layer despite the description. Uploads are now `cipher_version`
17
+ 3 (suite 0x03, ML-KEM-1024 + X25519 hybrid, v1 wire format) written by
18
+ `@shieldfive/crypto`'s stream encoder, with the row id as the header's
19
+ `file_id` and the filename re-sealed bound to the row (v6) at finalize. The
20
+ recipient key is derived locally from the root key, never fetched. Files
21
+ uploaded by earlier versions still open in the apps. Shared decrypt vectors:
22
+ `test/vectors/v3-upload.json`.
23
+ - `@shieldfive/crypto` is pinned to `~1.0.1` (was `^1.0.0-beta.1`, which a
24
+ global install could float to any 1.x).
25
+
26
+ ### Fixed
27
+
28
+ - `sf sync --watch` refreshes its session before the access token expires.
29
+ It signed in once, so after about an hour every pass failed every changed
30
+ file while the process kept running. If the session cannot be refreshed, sync
31
+ now stops with an error. A single `sf sync` pass with failed files exits
32
+ non-zero.
33
+
34
+ - **Sign-in works for accounts on ShieldFive's current sign-in scheme.** 0.3.0
35
+ posted your password straight to Supabase, which fails for every account that
36
+ signs in with a client-derived login secret (all new accounts, and every
37
+ account migrated on its next web or app sign-in), and sent the password to
38
+ the server on each failed attempt. `sf login`, `sf push` and `sf sync` now ask
39
+ ShieldFive how the account signs in, derive the login secret locally
40
+ (Argon2id, the same parameters and bytes as the web and Android apps) and
41
+ send only that. The raw password is sent only for an account the server
42
+ reports as not yet migrated, and only to ShieldFive's sign-in route. If that
43
+ lookup fails for any reason, sign-in stops before anything is sent.
44
+ - A vault password typed with leading or trailing spaces now unlocks a vault
45
+ created on the web, which trims them.
46
+
47
+ ### Changed
48
+
49
+ - Argon2id runs natively (the `argon2` package, off the main thread) when it
50
+ installs, and falls back to the bundled WebAssembly implementation when it
51
+ does not, so installation never fails on a platform without a prebuilt
52
+ binary. Both produce identical keys; parameters are unchanged.
9
53
 
10
54
  ## 0.3.0 - 2026-09-16
11
55
 
package/README.md CHANGED
@@ -11,8 +11,9 @@
11
11
 
12
12
  The point of this tool being open source is that you do not have to trust a
13
13
  marketing claim. The plaintext of your files is never sent to a ShieldFive
14
- server. The only bytes that leave your machine are AES-GCM ciphertext and a
15
- short cryptographic proof. Two files carry the whole story:
14
+ server. The only bytes that leave your machine are ciphertext (post-quantum
15
+ hybrid: ML-KEM-1024 + X25519 key encapsulation, AES-256-GCM chunks) and a short
16
+ cryptographic proof. Two files carry the whole story:
16
17
 
17
18
  - [`src/uploadCrypto.mjs`](src/uploadCrypto.mjs) — the encryption applied to
18
19
  every chunk and to the filename.
@@ -84,11 +85,18 @@ sf push <folder> encrypt and upload every file once
84
85
  sf sync <folder> [--watch] [--interval=N] upload new/changed files; --watch keeps a poll loop (N seconds, default 5)
85
86
  sf verify <file>... report whether each file is safely stored in your vault
86
87
  sf encrypt <folder> encrypt each file locally, upload nothing (no account needed)
88
+ sf inspect <folder> read-only recursive migration inventory; upload nothing
87
89
  ```
88
90
 
89
91
  `sf push` and `sf sync` pick the direct or multipart upload path automatically
90
92
  from the file size.
91
93
 
94
+ `sf inspect` reports JSON counts of files, directories, bytes and skipped entries.
95
+ It includes nested/hidden files and empty folders, reads metadata only, and
96
+ does not follow symbolic links. Exit code 2 means entries were skipped. It does
97
+ not upload anything or prove that a migration has completed. Current push/sync
98
+ remain top-level only; do not use their completion as proof of a whole-tree copy.
99
+
92
100
  With an agent running, `sf sync` records uploads in the agent's ledger (see
93
101
  [What the agent can and cannot do](#what-the-agent-can-and-cannot-do)) and
94
102
  `sf verify` works. Without one, `sf sync` writes a `.shieldfive-sync.json`
@@ -216,24 +224,36 @@ The full design, including the threat model, is in
216
224
 
217
225
  ## How the upload works (for auditors)
218
226
 
219
- The upload uses **chunked AES-GCM** for file content and issues a
220
- server-verified HMAC proof. (The `sf encrypt` demo uses the post-quantum hybrid
221
- suite; the sync/push path uses AES-GCM, which is what the vault's upload
222
- protocol and server-side proof expect.)
227
+ Uploads are written in the format the web app writes by default:
228
+ `cipher_version` 3, suite `0x03` (post-quantum hybrid) in the v1 wire format
229
+ (`@shieldfive/crypto` `spec/format-v1.md`), using the library's own stream
230
+ encoder.
223
231
 
224
232
  - **Filename** — encrypted with AES-GCM under a key derived from your vault root
225
- key via Argon2id (`encryptMetadataV4`). The server stores ciphertext; it never
226
- sees the name. A keyed HMAC of the lowercased name (`hashMetadataV4`) lets the
227
- server deduplicate without learning the name.
228
- - **File content** — each chunk is AES-GCM encrypted under a per-file content
229
- key (itself wrapped by your root key). The nonce is a 4-byte random prefix
230
- followed by an 8-byte big-endian chunk counter, so every chunk has a distinct
231
- nonce.
232
- - **Upload proof** — `HMAC-SHA256(proofKey, prefix || ciphertext)` over the first
233
- chunk, where `prefix = [version=1][cipherVersion=1][chunkSize u32 BE][noncePrefix 4]`.
234
- The server issues `proofKey` when it creates the session and verifies the proof
235
- when finalizing. It ties the stored ciphertext to the session without the
236
- server ever seeing plaintext.
233
+ key via Argon2id (`encryptMetadataV4`), then, at finalize, re-sealed with the
234
+ file's row id as additional data (v6, `encryptNameV6`), so the server cannot
235
+ move one file's name onto another row. A keyed HMAC of the lowercased name
236
+ (`hashMetadataV4`) lets the server deduplicate without learning the name.
237
+ - **File content** — a MAC'd header carries the file's `file_id` (the row id the
238
+ server assigned, so the stored object is bound to its row), the plaintext size
239
+ and the chunk count. The content key combines a per-file envelope key (wrapped
240
+ by your root key in `csk_wrapped`) with an ML-KEM-1024 + X25519 encapsulation
241
+ to your vault identity, which is derived on this machine from your root key,
242
+ as the web app derives it; the server cannot substitute a recipient. Each 5 MiB
243
+ chunk is AES-256-GCM with its index, the total and an is-final flag in the
244
+ additional data, so a dropped, reordered or truncated chunk fails to decrypt.
245
+ - **Upload proof** — base64 of `[3][3] || HMAC-SHA256(proofKey, [3][3] || header
246
+ || frame_0)` (`buildUploadProofV3` in `@shieldfive/crypto`, the same function
247
+ the web and the MCP server use). The server issues `proofKey` when it creates
248
+ the session and verifies the proof against the stored object when finalizing.
249
+ - **Parts** — each storage part is exactly one encrypted chunk frame; part 1 also
250
+ carries the header.
251
+
252
+ Versions up to 0.3.0 wrote the legacy v0 format (AES-GCM chunks under a random
253
+ nonce prefix, no header): no truncation detection, no row binding and no
254
+ post-quantum layer. Files already uploaded that way still open in the apps;
255
+ this client no longer writes it.
256
+
237
257
  - **Direct** (≤ 5 MiB) — the single encrypted chunk is `PUT` at a presigned S3
238
258
  URL the server issues with the session. The URL's SigV4 signature is the only
239
259
  credential and it authorises exactly one object key, so the request carries no
@@ -242,25 +262,33 @@ protocol and server-side proof expect.)
242
262
  Because the signature covers `host` only, nothing on this path binds the body:
243
263
  the server compares the client's `ciphertextHash` against the client's own
244
264
  `partSha1Array[0]`, so they always agree. Integrity here rests on the upload
245
- proof and on AES-GCM's tag at download, not on a wire checksum.
265
+ proof and on the header MAC and chunk tags at download, not on a wire checksum.
246
266
  - **Multipart** (> 5 MiB) — the file is streamed one 5 MiB chunk at a time (never
247
267
  loaded whole into memory), each chunk uploaded as a Backblaze part. The
248
268
  ciphertext hash the server checks is SHA-1 over the concatenated raw part
249
269
  digests; the server finishes the large file with the ordered part list.
250
270
 
251
271
  The mock end-to-end test in [`test/upload.flow.test.mjs`](test/upload.flow.test.mjs)
252
- captures every byte this client would send, decrypts each part with the key
253
- recovered from the session envelope, and asserts the result equals the original
254
- file — a machine-checked demonstration that only ciphertext is uploaded.
272
+ captures every byte this client would send, opens the stored object with
273
+ `@shieldfive/crypto`'s reference reader from nothing but the session fields and
274
+ the root key, and asserts the result equals the original file and the header's
275
+ `file_id` equals the row id. [`test/v1Format.test.mjs`](test/v1Format.test.mjs)
276
+ checks the shared vectors in [`test/vectors/v3-upload.json`](test/vectors/v3-upload.json)
277
+ (regenerate with `node scripts/generate-vectors.mjs`) against the library's
278
+ blob and streaming readers.
255
279
 
256
280
  ## Layout
257
281
 
258
282
  - `src/cli.mjs` — the `sf` entry point (argument parsing, command dispatch)
259
- - `src/auth.mjs` — Supabase sign-in to a Bearer token
283
+ - `src/auth.mjs` — sign-in to a Bearer token through ShieldFive's sign-in route,
284
+ with two-factor step-up
285
+ - `src/loginSecret.mjs` — the login secret derived from your password; for
286
+ accounts on the current scheme the password itself is never sent
287
+ - `src/argon2.mjs` — Argon2id, native when available, WebAssembly otherwise
260
288
  - `src/vault.mjs` / `src/unlock.mjs` — fetch the wrapped vault key, unwrap the
261
289
  root key from your password (Argon2id / PBKDF2)
262
- - `src/uploadCrypto.mjs` — filename and chunk encryption, upload proof, multipart
263
- ciphertext hash
290
+ - `src/uploadCrypto.mjs` — filename encryption, the suite-0x03 part writer,
291
+ upload proof, multipart ciphertext hash
264
292
  - `src/upload.mjs` — create session, encrypt, upload (direct + multipart),
265
293
  finalize; the streaming chunk reader
266
294
  - `src/sync.mjs` — `sf sync`: manifest, change detection, reconcile pass, watch
package/package.json CHANGED
@@ -1,8 +1,8 @@
1
1
  {
2
2
  "name": "@shieldfive/cli",
3
- "version": "0.3.0",
3
+ "version": "0.3.1",
4
4
  "type": "module",
5
- "description": "Open-source command-line client for ShieldFive: encrypt a folder on your machine and sync it to your EU-hosted, post-quantum vault. Read the code; it encrypts before upload.",
5
+ "description": "Open-source command-line client for ShieldFive: encrypt a folder on your machine and sync it to your EU-hosted vault with post-quantum hybrid encryption (ML-KEM-1024 + X25519). Read the code; it encrypts before upload.",
6
6
  "license": "Apache-2.0",
7
7
  "author": "Cho Garcia <security@shieldfive.com>",
8
8
  "homepage": "https://github.com/shieldfive/cli",
@@ -46,8 +46,11 @@
46
46
  "dependencies": {
47
47
  "@noble/ciphers": "^2.1.1",
48
48
  "@noble/hashes": "^2.0.1",
49
- "@shieldfive/crypto": "^1.0.0-beta.1",
49
+ "@shieldfive/crypto": "~1.0.1",
50
50
  "@supabase/supabase-js": "^2",
51
51
  "libsodium-wrappers-sumo": "^0.8.2"
52
+ },
53
+ "optionalDependencies": {
54
+ "argon2": "0.45.1"
52
55
  }
53
56
  }
package/src/argon2.mjs ADDED
@@ -0,0 +1,69 @@
1
+ import { hkdfSync } from 'node:crypto'
2
+
3
+ // Argon2id master-secret derivation, byte-identical to
4
+ // @shieldfive/crypto/kdf/argon2id deriveMasterSecret (the web's and the
5
+ // vault's reference implementation): libsodium presets, Argon2id v0x13,
6
+ // parallelism 1, 32-byte output, and salts longer than 16 bytes compressed
7
+ // through HKDF-SHA-256 (zero salt, info 'shieldfive/v1/argon2id/salt-compression').
8
+ //
9
+ // Two backends, same bytes:
10
+ //
11
+ // - native `argon2` (an optionalDependency): async, runs on the libuv thread
12
+ // pool so the event loop keeps turning, and several times faster.
13
+ // - @shieldfive/crypto over libsodium WASM: always installed. Used when the
14
+ // native module is missing, which is expected on platforms `argon2` ships no
15
+ // prebuilt binary for (Intel macOS, Windows on ARM) and where compiling it
16
+ // failed. npm drops a failed optional dependency instead of failing the
17
+ // install, so `npm i -g @shieldfive/cli` never hard-fails on it.
18
+ //
19
+ // Parameters are never reduced on either path. test/argon2.backends.test.mjs
20
+ // pins the two backends against each other and against web-produced vectors.
21
+
22
+ const PRESETS = {
23
+ moderate: { timeCost: 3, memoryCost: 262144 }, // OPSLIMIT/MEMLIMIT_MODERATE (256 MiB)
24
+ sensitive: { timeCost: 4, memoryCost: 1048576 }, // OPSLIMIT/MEMLIMIT_SENSITIVE (1 GiB)
25
+ }
26
+
27
+ let nativePromise
28
+ async function loadNative() {
29
+ if (process.env.SF_ARGON2_BACKEND === 'wasm') return null
30
+ nativePromise ??= import('argon2').then(
31
+ (m) => (typeof m.hash === 'function' ? m : m.default ?? null),
32
+ () => null,
33
+ )
34
+ return nativePromise
35
+ }
36
+
37
+ /** 'native' or 'wasm': which backend the next derivation will use. */
38
+ export async function argon2Backend() {
39
+ return (await loadNative()) ? 'native' : 'wasm'
40
+ }
41
+
42
+ function validate({ passphrase, salt, preset }) {
43
+ if (typeof passphrase !== 'string' || !passphrase) throw new Error('Passphrase is required')
44
+ if (!(salt instanceof Uint8Array) || salt.length < 16) throw new Error('Argon2id salt must be at least 16 bytes')
45
+ const params = Object.hasOwn(PRESETS, preset) ? PRESETS[preset] : null
46
+ if (!params) throw new Error(`Unsupported Argon2id preset: ${preset}`)
47
+ return params
48
+ }
49
+
50
+ export async function deriveMasterSecret({ passphrase, salt, preset = 'moderate' }) {
51
+ const params = validate({ passphrase, salt, preset })
52
+ const native = await loadNative()
53
+ if (!native) {
54
+ const { deriveMasterSecret: wasmDerive } = await import('@shieldfive/crypto/kdf/argon2id')
55
+ const { masterSecret } = await wasmDerive({ passphrase, salt, preset })
56
+ return { masterSecret, salt, preset }
57
+ }
58
+ const usableSalt = salt.length === 16 ? Buffer.from(salt) : Buffer.from(hkdfSync(
59
+ 'sha256', salt, Buffer.alloc(32), 'shieldfive/v1/argon2id/salt-compression', 16,
60
+ ))
61
+ const input = Buffer.from(passphrase, 'utf8')
62
+ try {
63
+ const masterSecret = await native.hash(input, {
64
+ ...params, salt: usableSalt, type: native.argon2id, version: 0x13,
65
+ parallelism: 1, hashLength: 32, raw: true,
66
+ })
67
+ return { masterSecret, salt, preset } // a Buffer, i.e. a Uint8Array the caller zeroes
68
+ } finally { input.fill(0); usableSalt.fill(0) }
69
+ }
package/src/auth.mjs CHANGED
@@ -1,6 +1,7 @@
1
1
  // ShieldFive CLI — authentication.
2
2
  //
3
- // Signs in via Supabase to obtain a Bearer access token, elevating to AAL2 when
3
+ // Signs in through ShieldFive's version-aware login route to obtain a Bearer
4
+ // session, then installs it in Supabase and elevates to AAL2 when
4
5
  // the account has two-factor authentication (TOTP) enabled. The upload routes
5
6
  // accept this token, and their server-side MFA gate requires an AAL2 token for
6
7
  // enrolled accounts — so the step-up here is what makes `sf push` / `sf sync`
@@ -11,6 +12,7 @@
11
12
  // via getAuthenticatorAssuranceLevel.
12
13
 
13
14
  import { createClient } from '@supabase/supabase-js'
15
+ import { buildSignInCredentials } from './loginSecret.mjs'
14
16
 
15
17
  // If the account requires an AAL2 step-up, run the TOTP challenge/verify flow
16
18
  // and return the elevated access token. Returns null when no step-up is needed
@@ -77,24 +79,48 @@ export async function stepUpMfaIfRequired({ supabase, getTotpCode }) {
77
79
  }
78
80
 
79
81
  export async function signIn({
82
+ apiBaseUrl,
80
83
  supabaseUrl,
81
84
  anonKey,
82
85
  email,
83
86
  password,
84
87
  getTotpCode,
88
+ fetchImpl = fetch,
89
+ createClientImpl = createClient,
85
90
  }) {
86
- const supabase = createClient(supabaseUrl, anonKey, {
91
+ // The raw password is never handed to Supabase directly. The salt lookup
92
+ // decides what the version-aware login route receives: only the derived
93
+ // login secret for a version-2 account; the password only for an account the
94
+ // server affirmatively reported as version 1. test/auth.noPasswordLeak.test.mjs
95
+ // pins this across every path.
96
+ email = String(email ?? '').trim().toLowerCase()
97
+ const credentials = await buildSignInCredentials({ apiBaseUrl, email, password, fetchImpl })
98
+ let response
99
+ try {
100
+ response = await fetchImpl(new URL('/api/mobile/auth/login', apiBaseUrl), {
101
+ method: 'POST', headers: { 'Content-Type': 'application/json', Accept: 'application/json' },
102
+ body: JSON.stringify({ email, ...credentials }), signal: AbortSignal.timeout(30_000),
103
+ })
104
+ } catch {
105
+ throw new Error('Sign-in failed: could not reach ShieldFive. Check your connection and try again.')
106
+ }
107
+ const payload = await response.json().catch(() => null)
108
+ if (!response.ok || typeof payload?.accessToken !== 'string' || !payload.accessToken ||
109
+ typeof payload?.refreshToken !== 'string' || !payload.refreshToken) {
110
+ throw new Error(response.status === 429 ? 'Too many sign-in attempts. Try again shortly.' : 'Sign-in failed. Check your credentials and try again.')
111
+ }
112
+ const supabase = createClientImpl(supabaseUrl, anonKey, {
87
113
  auth: {
88
114
  persistSession: false,
89
115
  autoRefreshToken: false,
90
116
  detectSessionInUrl: false,
91
117
  },
92
118
  })
93
- const { data, error } = await supabase.auth.signInWithPassword({
94
- email,
95
- password,
119
+ const { data, error } = await supabase.auth.setSession({
120
+ access_token: payload.accessToken,
121
+ refresh_token: payload.refreshToken,
96
122
  })
97
- if (error) throw new Error(`Sign-in failed: ${error.message}`)
123
+ if (error) throw new Error('Could not establish the sign-in session. Please try again.')
98
124
 
99
125
  const aal1Token = data.session?.access_token
100
126
  if (!aal1Token) {
package/src/cli.mjs CHANGED
@@ -35,6 +35,7 @@ function usage() {
35
35
  ' [--interval=<seconds>] poll interval for --watch (default 5)',
36
36
  ' verify <file>... report whether each file is safely in your vault',
37
37
  ' encrypt <folder> encrypt each file locally, upload nothing',
38
+ ' inspect <folder> count a migration source recursively; upload nothing',
38
39
  ' --help',
39
40
  '',
40
41
  'After sf login, push / sync / verify go through the agent and need nothing',
@@ -143,7 +144,8 @@ async function authAndUnlock(cfg) {
143
144
  const { fetchAndUnlockVault } = await import('./vault.mjs')
144
145
 
145
146
  process.stderr.write('signing in…\n')
146
- const { accessToken } = await signIn({
147
+ const { accessToken, refreshToken, expiresAt } = await signIn({
148
+ apiBaseUrl: cfg.apiBaseUrl,
147
149
  supabaseUrl: cfg.supabaseUrl,
148
150
  anonKey: cfg.anonKey,
149
151
  email: cfg.email,
@@ -157,7 +159,13 @@ async function authAndUnlock(cfg) {
157
159
  accessToken,
158
160
  password: cfg.vaultPassword,
159
161
  })
160
- return { accessToken, rootKey }
162
+ // `sf sync --watch` outlives the access token (about an hour); it refreshes
163
+ // through this, the same exchange the agent uses.
164
+ const refreshSession = async (token) => {
165
+ const { refreshAccessToken } = await import('./auth.mjs')
166
+ return refreshAccessToken({ supabaseUrl: cfg.supabaseUrl, anonKey: cfg.anonKey, refreshToken: token })
167
+ }
168
+ return { accessToken, refreshToken, expiresAt, refreshSession, rootKey }
161
169
  }
162
170
 
163
171
  async function runningAgent() {
@@ -540,6 +548,12 @@ async function main() {
540
548
  } else if (cmd === 'encrypt') {
541
549
  if (!folder) throw new Error('encrypt needs a <folder>')
542
550
  await cmdEncrypt(folder)
551
+ } else if (cmd === 'inspect') {
552
+ if (!folder) throw new Error('inspect needs a <folder>')
553
+ const { inspectMigrationSource } = await import('./migrationInventory.mjs')
554
+ const result = await inspectMigrationSource(folder)
555
+ process.stdout.write(JSON.stringify(result) + '\n')
556
+ if (result.skipped) process.exitCode = 2
543
557
  } else if (cmd === 'push') {
544
558
  if (!folder) throw new Error('push needs a <folder>')
545
559
  await cmdPush(folder)
@@ -0,0 +1,63 @@
1
+ // Client-side derivation of the LOGIN SECRET, byte-identical to the web
2
+ // (utils/auth/loginSecret.ts) and Android (src/features/auth/loginSecret.ts):
3
+ //
4
+ // loginSecret = base64url_unpadded(
5
+ // Argon2id( DOMAIN || "\n" || password, authSalt, moderate ) )
6
+ //
7
+ // The password is used EXACTLY as typed: no trim, no Unicode normalisation.
8
+ // That is what web sign-in, signup, reset and change-password all feed this
9
+ // derivation, so it is what the secret GoTrue holds was derived from. (The
10
+ // vault KDF is different: the web keyring trims the vault password. See
11
+ // src/unlock.mjs.)
12
+ import { deriveMasterSecret } from './argon2.mjs'
13
+
14
+ const DOMAIN = 'shieldfive/v1/auth/login-secret'
15
+
16
+ export async function deriveLoginSecret({ password, authSalt }) {
17
+ if (typeof password !== 'string' || !password) throw new Error('Password is required')
18
+ if (typeof authSalt !== 'string' || !/^[A-Za-z0-9+/_-]+={0,2}$/.test(authSalt)) {
19
+ throw new Error('Invalid secure sign-in salt')
20
+ }
21
+ // Node's base64 decoder accepts base64 and base64url alike, as the web does.
22
+ const salt = new Uint8Array(Buffer.from(authSalt, 'base64'))
23
+ if (salt.length < 16 || salt.length > 64) throw new Error('Invalid secure sign-in salt')
24
+ const { masterSecret } = await deriveMasterSecret({
25
+ passphrase: `${DOMAIN}\n${password}`, salt, preset: 'moderate',
26
+ })
27
+ try { return Buffer.from(masterSecret).toString('base64url') }
28
+ finally { masterSecret.fill(0) }
29
+ }
30
+
31
+ /**
32
+ * The credential half of the sign-in body, per POST /api/auth/salt:
33
+ *
34
+ * version 2 -> { loginSecret } (no password)
35
+ * version 1, migration on -> { password, loginSecret, authSalt } (server verifies, then migrates)
36
+ * version 1, migration off -> { password }
37
+ *
38
+ * The raw password is only ever returned when the server AFFIRMATIVELY said
39
+ * version 1. Unlike the web's fetchAuthSalt, a failed, throttled or malformed
40
+ * lookup never falls back to version 1: for a migrated account that would post
41
+ * the password to the server and still not sign in. Throws before anything is
42
+ * posted to the login route.
43
+ */
44
+ export async function buildSignInCredentials({ apiBaseUrl, email, password, fetchImpl = fetch }) {
45
+ let response
46
+ try {
47
+ response = await fetchImpl(new URL('/api/auth/salt', apiBaseUrl), {
48
+ method: 'POST', headers: { 'Content-Type': 'application/json', Accept: 'application/json' },
49
+ body: JSON.stringify({ email }), signal: AbortSignal.timeout(30_000),
50
+ })
51
+ } catch {
52
+ throw new Error('Unable to prepare secure sign-in. Check your connection and try again.')
53
+ }
54
+ if (response.status === 429) throw new Error('Too many sign-in attempts. Try again shortly.')
55
+ const body = await response.json().catch(() => null)
56
+ if (!response.ok || !body || ![1, 2].includes(body.authVersion) ||
57
+ typeof body.authSalt !== 'string' || !body.authSalt) {
58
+ throw new Error('Unable to prepare secure sign-in. Please try again.')
59
+ }
60
+ if (body.authVersion === 1 && body.migrationEnabled !== true) return { password }
61
+ const loginSecret = await deriveLoginSecret({ password, authSalt: body.authSalt })
62
+ return body.authVersion === 2 ? { loginSecret } : { password, loginSecret, authSalt: body.authSalt }
63
+ }
@@ -0,0 +1,45 @@
1
+ import { lstat, opendir } from 'node:fs/promises'
2
+ import { join, relative, resolve, sep } from 'node:path'
3
+
4
+ // Read-only preflight: includes hidden files and empty directories, preserves
5
+ // names, and reports unsupported entries instead of silently omitting them.
6
+ // It streams metadata without reading file contents or holding an inventory in
7
+ // memory. Upload must revalidate each source: this is not an immutable snapshot.
8
+ export async function* inventoryFolder(folder) {
9
+ const root = resolve(folder)
10
+ const initial = await lstat(root)
11
+ if (!initial.isDirectory() || initial.isSymbolicLink()) throw new Error('Select a real source folder, not a symbolic link')
12
+ const pending = [root]
13
+ while (pending.length) {
14
+ const directory = pending.pop()
15
+ const directoryInfo = await lstat(directory)
16
+ if (directoryInfo.isSymbolicLink() || !directoryInfo.isDirectory()) {
17
+ yield { kind: 'skipped', path: relative(root, directory).split(sep).join('/'), reason: 'source_changed' }
18
+ continue
19
+ }
20
+ for await (const entry of await opendir(directory)) {
21
+ const absolutePath = join(directory, entry.name)
22
+ const path = relative(root, absolutePath).split(sep).join('/')
23
+ let info
24
+ try { info = await lstat(absolutePath) }
25
+ catch (error) { yield { kind: 'skipped', path, reason: error.code ?? 'unreadable' }; continue }
26
+ if (info.isSymbolicLink()) yield { kind: 'skipped', path, reason: 'symbolic_link' }
27
+ else if (info.isDirectory()) {
28
+ yield { kind: 'directory', path }
29
+ pending.push(absolutePath)
30
+ } else if (info.isFile()) yield { kind: 'file', path, size: info.size, mtimeMs: info.mtimeMs }
31
+ else yield { kind: 'skipped', path, reason: 'unsupported_file_type' }
32
+ }
33
+ }
34
+ }
35
+
36
+ export async function inspectMigrationSource(folder) {
37
+ const totals = { files: 0, directories: 0, bytes: 0, skipped: 0 }
38
+ for await (const entry of inventoryFolder(folder)) {
39
+ if (entry.kind === 'file') { totals.files++; totals.bytes += entry.size }
40
+ else if (entry.kind === 'directory') totals.directories++
41
+ else totals.skipped++
42
+ if (!Number.isSafeInteger(totals.bytes)) throw new Error('Source size exceeds the supported inventory limit')
43
+ }
44
+ return totals
45
+ }
package/src/sync.mjs CHANGED
@@ -11,9 +11,11 @@
11
11
  // Scope notes: change is detected by (size, mtimeMs) — the rsync default, cheap
12
12
  // and good enough; a same-size same-mtime edit is not re-uploaded. Deletions
13
13
  // and renames are NOT mirrored (a file removed locally stays in the vault); sync
14
- // is append-only for now. Each changed file becomes a new vault upload (the
15
- // server does not overwrite by name), which is the intended versioning
16
- // behaviour.
14
+ // is append-only for now. Each changed file becomes a new vault upload at the
15
+ // vault root (the server does not overwrite by name); earlier versions are kept,
16
+ // not replaced or trashed, so they count against storage until removed in the
17
+ // app. Replacing the previous fileId needs the Bin's folder key to re-wrap it,
18
+ // which this client does not hold yet.
17
19
 
18
20
  import { open, readFile, readdir, rename, stat } from 'node:fs/promises'
19
21
  import { join } from 'node:path'
@@ -95,6 +97,11 @@ export function planSync(entries, manifest) {
95
97
  // One reconcile pass. uploadFn is injected (defaults to the real uploadFile) so
96
98
  // the pass is unit-testable without a network. Returns a summary and mutates
97
99
  // `manifest` in place, persisting after each successful upload.
100
+ //
101
+ // `accessToken` is a string, or a function returning one; the function is
102
+ // called before every upload, so a long pass picks up a refreshed session. If
103
+ // it throws (the session cannot be refreshed) the pass stops and the error
104
+ // propagates: nothing after it could succeed.
98
105
  export async function syncOnce({
99
106
  apiBaseUrl,
100
107
  accessToken,
@@ -112,11 +119,13 @@ export async function syncOnce({
112
119
  let failed = 0
113
120
  const errors = []
114
121
 
122
+ const tokenFor = typeof accessToken === 'function' ? accessToken : async () => accessToken
115
123
  for (const entry of toUpload) {
124
+ const token = await tokenFor()
116
125
  try {
117
126
  const { fileId } = await uploadFn({
118
127
  apiBaseUrl,
119
- accessToken,
128
+ accessToken: token,
120
129
  rootKey,
121
130
  name: entry.name,
122
131
  path: entry.path,
@@ -156,9 +165,47 @@ function sleep(ms, signal) {
156
165
  })
157
166
  }
158
167
 
168
+ // Refresh this long before the access token expires (seconds).
169
+ const REFRESH_MARGIN_S = 120
170
+
171
+ /**
172
+ * The access token for the next request: the current one while it has more
173
+ * than REFRESH_MARGIN_S left, otherwise a refreshed one. Supabase rotates
174
+ * refresh tokens and revokes the session if one is presented twice, so the
175
+ * refresh is single-flight and the rotated token replaces the old one.
176
+ * Without an expiry or a refresh function (tests, old callers) the token is
177
+ * used as is.
178
+ */
179
+ export function createSessionTokens({ accessToken, refreshToken, expiresAt, refresh, now = () => Date.now() }) {
180
+ let tokens = { access: accessToken, refresh: refreshToken, expiresAt }
181
+ let inflight = null
182
+ return async function current() {
183
+ if (!refresh || !tokens.expiresAt || now() / 1000 < tokens.expiresAt - REFRESH_MARGIN_S) {
184
+ return tokens.access
185
+ }
186
+ inflight ??= refresh(tokens.refresh)
187
+ .then((s) => {
188
+ tokens = { access: s.accessToken, refresh: s.refreshToken, expiresAt: s.expiresAt }
189
+ return tokens.access
190
+ })
191
+ .finally(() => {
192
+ inflight = null
193
+ })
194
+ return inflight
195
+ }
196
+ }
197
+
159
198
  // Orchestrates a real sync: sign in + unlock once, then run one pass (or loop on
160
199
  // an interval under --watch). authAndUnlock is injected so the CLI can share its
161
- // sign-in/unlock path and tests can stub it.
200
+ // sign-in/unlock path and tests can stub it. It may also return refreshToken,
201
+ // expiresAt (unix seconds) and refreshSession(refreshToken); with them the
202
+ // session is refreshed before it expires, so `--watch` keeps working past the
203
+ // access token's lifetime (about an hour). A refresh that fails ends the sync
204
+ // with an error (non-zero exit) instead of failing every file on every pass
205
+ // while looking healthy to a supervisor.
206
+ //
207
+ // Returns the last pass's summary. A single pass with failed files rejects, so
208
+ // `sf sync` exits non-zero; under --watch failed files are retried next pass.
162
209
  export async function runSync({
163
210
  cfg,
164
211
  folder,
@@ -168,8 +215,17 @@ export async function runSync({
168
215
  uploadFn,
169
216
  log = (m) => process.stdout.write(`${m}\n`),
170
217
  status = (m) => process.stderr.write(`${m}\n`),
218
+ now = () => Date.now(),
171
219
  }) {
172
- const { accessToken, rootKey } = await authAndUnlock(cfg)
220
+ const session = await authAndUnlock(cfg)
221
+ const { rootKey } = session
222
+ const accessToken = createSessionTokens({
223
+ accessToken: session.accessToken,
224
+ refreshToken: session.refreshToken,
225
+ expiresAt: session.expiresAt,
226
+ refresh: session.refreshSession,
227
+ now,
228
+ })
173
229
  const manifestPath = join(folder, MANIFEST_NAME)
174
230
  const manifest = await loadManifest(manifestPath)
175
231
 
@@ -180,9 +236,10 @@ export async function runSync({
180
236
  }
181
237
  if (watch) process.on('SIGINT', onSigint)
182
238
 
239
+ let summary
183
240
  try {
184
241
  do {
185
- const summary = await syncOnce({
242
+ summary = await syncOnce({
186
243
  apiBaseUrl: cfg.apiBaseUrl,
187
244
  accessToken,
188
245
  rootKey,
@@ -202,4 +259,8 @@ export async function runSync({
202
259
  } finally {
203
260
  if (watch) process.off('SIGINT', onSigint)
204
261
  }
262
+ if (!watch && summary?.failed) {
263
+ throw new Error(`${summary.failed} file(s) failed to sync.`)
264
+ }
265
+ return summary
205
266
  }