@shieldfive/cli 0.3.0 → 0.3.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +45 -1
- package/README.md +53 -25
- package/package.json +6 -3
- package/src/argon2.mjs +69 -0
- package/src/auth.mjs +32 -6
- package/src/cli.mjs +16 -2
- package/src/loginSecret.mjs +63 -0
- package/src/migrationInventory.mjs +45 -0
- package/src/sync.mjs +68 -7
- package/src/unlock.mjs +32 -16
- package/src/upload.mjs +103 -68
- package/src/uploadCrypto.mjs +153 -113
package/CHANGELOG.md
CHANGED
|
@@ -5,7 +5,51 @@ All notable changes to `@shieldfive/cli` will be documented in this file.
|
|
|
5
5
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
|
6
6
|
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
7
|
|
|
8
|
-
##
|
|
8
|
+
## 0.3.1 - 2026-10-01
|
|
9
|
+
|
|
10
|
+
### Security
|
|
11
|
+
|
|
12
|
+
- **`sf push` and `sf sync` write the web's default format.** They wrote the
|
|
13
|
+
deprecated v0 format (AES-GCM chunks, no header), which the format spec
|
|
14
|
+
forbids for new writes: a server could drop trailing chunks and the file
|
|
15
|
+
still decrypted, could move a file's bytes onto another row, and there was no
|
|
16
|
+
post-quantum layer despite the description. Uploads are now `cipher_version`
|
|
17
|
+
3 (suite 0x03, ML-KEM-1024 + X25519 hybrid, v1 wire format) written by
|
|
18
|
+
`@shieldfive/crypto`'s stream encoder, with the row id as the header's
|
|
19
|
+
`file_id` and the filename re-sealed bound to the row (v6) at finalize. The
|
|
20
|
+
recipient key is derived locally from the root key, never fetched. Files
|
|
21
|
+
uploaded by earlier versions still open in the apps. Shared decrypt vectors:
|
|
22
|
+
`test/vectors/v3-upload.json`.
|
|
23
|
+
- `@shieldfive/crypto` is pinned to `~1.0.1` (was `^1.0.0-beta.1`, which a
|
|
24
|
+
global install could float to any 1.x).
|
|
25
|
+
|
|
26
|
+
### Fixed
|
|
27
|
+
|
|
28
|
+
- `sf sync --watch` refreshes its session before the access token expires.
|
|
29
|
+
It signed in once, so after about an hour every pass failed every changed
|
|
30
|
+
file while the process kept running. If the session cannot be refreshed, sync
|
|
31
|
+
now stops with an error. A single `sf sync` pass with failed files exits
|
|
32
|
+
non-zero.
|
|
33
|
+
|
|
34
|
+
- **Sign-in works for accounts on ShieldFive's current sign-in scheme.** 0.3.0
|
|
35
|
+
posted your password straight to Supabase, which fails for every account that
|
|
36
|
+
signs in with a client-derived login secret (all new accounts, and every
|
|
37
|
+
account migrated on its next web or app sign-in), and sent the password to
|
|
38
|
+
the server on each failed attempt. `sf login`, `sf push` and `sf sync` now ask
|
|
39
|
+
ShieldFive how the account signs in, derive the login secret locally
|
|
40
|
+
(Argon2id, the same parameters and bytes as the web and Android apps) and
|
|
41
|
+
send only that. The raw password is sent only for an account the server
|
|
42
|
+
reports as not yet migrated, and only to ShieldFive's sign-in route. If that
|
|
43
|
+
lookup fails for any reason, sign-in stops before anything is sent.
|
|
44
|
+
- A vault password typed with leading or trailing spaces now unlocks a vault
|
|
45
|
+
created on the web, which trims them.
|
|
46
|
+
|
|
47
|
+
### Changed
|
|
48
|
+
|
|
49
|
+
- Argon2id runs natively (the `argon2` package, off the main thread) when it
|
|
50
|
+
installs, and falls back to the bundled WebAssembly implementation when it
|
|
51
|
+
does not, so installation never fails on a platform without a prebuilt
|
|
52
|
+
binary. Both produce identical keys; parameters are unchanged.
|
|
9
53
|
|
|
10
54
|
## 0.3.0 - 2026-09-16
|
|
11
55
|
|
package/README.md
CHANGED
|
@@ -11,8 +11,9 @@
|
|
|
11
11
|
|
|
12
12
|
The point of this tool being open source is that you do not have to trust a
|
|
13
13
|
marketing claim. The plaintext of your files is never sent to a ShieldFive
|
|
14
|
-
server. The only bytes that leave your machine are
|
|
15
|
-
|
|
14
|
+
server. The only bytes that leave your machine are ciphertext (post-quantum
|
|
15
|
+
hybrid: ML-KEM-1024 + X25519 key encapsulation, AES-256-GCM chunks) and a short
|
|
16
|
+
cryptographic proof. Two files carry the whole story:
|
|
16
17
|
|
|
17
18
|
- [`src/uploadCrypto.mjs`](src/uploadCrypto.mjs) — the encryption applied to
|
|
18
19
|
every chunk and to the filename.
|
|
@@ -84,11 +85,18 @@ sf push <folder> encrypt and upload every file once
|
|
|
84
85
|
sf sync <folder> [--watch] [--interval=N] upload new/changed files; --watch keeps a poll loop (N seconds, default 5)
|
|
85
86
|
sf verify <file>... report whether each file is safely stored in your vault
|
|
86
87
|
sf encrypt <folder> encrypt each file locally, upload nothing (no account needed)
|
|
88
|
+
sf inspect <folder> read-only recursive migration inventory; upload nothing
|
|
87
89
|
```
|
|
88
90
|
|
|
89
91
|
`sf push` and `sf sync` pick the direct or multipart upload path automatically
|
|
90
92
|
from the file size.
|
|
91
93
|
|
|
94
|
+
`sf inspect` reports JSON counts of files, directories, bytes and skipped entries.
|
|
95
|
+
It includes nested/hidden files and empty folders, reads metadata only, and
|
|
96
|
+
does not follow symbolic links. Exit code 2 means entries were skipped. It does
|
|
97
|
+
not upload anything or prove that a migration has completed. Current push/sync
|
|
98
|
+
remain top-level only; do not use their completion as proof of a whole-tree copy.
|
|
99
|
+
|
|
92
100
|
With an agent running, `sf sync` records uploads in the agent's ledger (see
|
|
93
101
|
[What the agent can and cannot do](#what-the-agent-can-and-cannot-do)) and
|
|
94
102
|
`sf verify` works. Without one, `sf sync` writes a `.shieldfive-sync.json`
|
|
@@ -216,24 +224,36 @@ The full design, including the threat model, is in
|
|
|
216
224
|
|
|
217
225
|
## How the upload works (for auditors)
|
|
218
226
|
|
|
219
|
-
|
|
220
|
-
|
|
221
|
-
|
|
222
|
-
|
|
227
|
+
Uploads are written in the format the web app writes by default:
|
|
228
|
+
`cipher_version` 3, suite `0x03` (post-quantum hybrid) in the v1 wire format
|
|
229
|
+
(`@shieldfive/crypto` `spec/format-v1.md`), using the library's own stream
|
|
230
|
+
encoder.
|
|
223
231
|
|
|
224
232
|
- **Filename** — encrypted with AES-GCM under a key derived from your vault root
|
|
225
|
-
key via Argon2id (`encryptMetadataV4`)
|
|
226
|
-
|
|
227
|
-
|
|
228
|
-
|
|
229
|
-
|
|
230
|
-
|
|
231
|
-
|
|
232
|
-
|
|
233
|
-
|
|
234
|
-
|
|
235
|
-
|
|
236
|
-
|
|
233
|
+
key via Argon2id (`encryptMetadataV4`), then, at finalize, re-sealed with the
|
|
234
|
+
file's row id as additional data (v6, `encryptNameV6`), so the server cannot
|
|
235
|
+
move one file's name onto another row. A keyed HMAC of the lowercased name
|
|
236
|
+
(`hashMetadataV4`) lets the server deduplicate without learning the name.
|
|
237
|
+
- **File content** — a MAC'd header carries the file's `file_id` (the row id the
|
|
238
|
+
server assigned, so the stored object is bound to its row), the plaintext size
|
|
239
|
+
and the chunk count. The content key combines a per-file envelope key (wrapped
|
|
240
|
+
by your root key in `csk_wrapped`) with an ML-KEM-1024 + X25519 encapsulation
|
|
241
|
+
to your vault identity, which is derived on this machine from your root key,
|
|
242
|
+
as the web app derives it; the server cannot substitute a recipient. Each 5 MiB
|
|
243
|
+
chunk is AES-256-GCM with its index, the total and an is-final flag in the
|
|
244
|
+
additional data, so a dropped, reordered or truncated chunk fails to decrypt.
|
|
245
|
+
- **Upload proof** — base64 of `[3][3] || HMAC-SHA256(proofKey, [3][3] || header
|
|
246
|
+
|| frame_0)` (`buildUploadProofV3` in `@shieldfive/crypto`, the same function
|
|
247
|
+
the web and the MCP server use). The server issues `proofKey` when it creates
|
|
248
|
+
the session and verifies the proof against the stored object when finalizing.
|
|
249
|
+
- **Parts** — each storage part is exactly one encrypted chunk frame; part 1 also
|
|
250
|
+
carries the header.
|
|
251
|
+
|
|
252
|
+
Versions up to 0.3.0 wrote the legacy v0 format (AES-GCM chunks under a random
|
|
253
|
+
nonce prefix, no header): no truncation detection, no row binding and no
|
|
254
|
+
post-quantum layer. Files already uploaded that way still open in the apps;
|
|
255
|
+
this client no longer writes it.
|
|
256
|
+
|
|
237
257
|
- **Direct** (≤ 5 MiB) — the single encrypted chunk is `PUT` at a presigned S3
|
|
238
258
|
URL the server issues with the session. The URL's SigV4 signature is the only
|
|
239
259
|
credential and it authorises exactly one object key, so the request carries no
|
|
@@ -242,25 +262,33 @@ protocol and server-side proof expect.)
|
|
|
242
262
|
Because the signature covers `host` only, nothing on this path binds the body:
|
|
243
263
|
the server compares the client's `ciphertextHash` against the client's own
|
|
244
264
|
`partSha1Array[0]`, so they always agree. Integrity here rests on the upload
|
|
245
|
-
proof and on
|
|
265
|
+
proof and on the header MAC and chunk tags at download, not on a wire checksum.
|
|
246
266
|
- **Multipart** (> 5 MiB) — the file is streamed one 5 MiB chunk at a time (never
|
|
247
267
|
loaded whole into memory), each chunk uploaded as a Backblaze part. The
|
|
248
268
|
ciphertext hash the server checks is SHA-1 over the concatenated raw part
|
|
249
269
|
digests; the server finishes the large file with the ordered part list.
|
|
250
270
|
|
|
251
271
|
The mock end-to-end test in [`test/upload.flow.test.mjs`](test/upload.flow.test.mjs)
|
|
252
|
-
captures every byte this client would send,
|
|
253
|
-
|
|
254
|
-
|
|
272
|
+
captures every byte this client would send, opens the stored object with
|
|
273
|
+
`@shieldfive/crypto`'s reference reader from nothing but the session fields and
|
|
274
|
+
the root key, and asserts the result equals the original file and the header's
|
|
275
|
+
`file_id` equals the row id. [`test/v1Format.test.mjs`](test/v1Format.test.mjs)
|
|
276
|
+
checks the shared vectors in [`test/vectors/v3-upload.json`](test/vectors/v3-upload.json)
|
|
277
|
+
(regenerate with `node scripts/generate-vectors.mjs`) against the library's
|
|
278
|
+
blob and streaming readers.
|
|
255
279
|
|
|
256
280
|
## Layout
|
|
257
281
|
|
|
258
282
|
- `src/cli.mjs` — the `sf` entry point (argument parsing, command dispatch)
|
|
259
|
-
- `src/auth.mjs` —
|
|
283
|
+
- `src/auth.mjs` — sign-in to a Bearer token through ShieldFive's sign-in route,
|
|
284
|
+
with two-factor step-up
|
|
285
|
+
- `src/loginSecret.mjs` — the login secret derived from your password; for
|
|
286
|
+
accounts on the current scheme the password itself is never sent
|
|
287
|
+
- `src/argon2.mjs` — Argon2id, native when available, WebAssembly otherwise
|
|
260
288
|
- `src/vault.mjs` / `src/unlock.mjs` — fetch the wrapped vault key, unwrap the
|
|
261
289
|
root key from your password (Argon2id / PBKDF2)
|
|
262
|
-
- `src/uploadCrypto.mjs` — filename
|
|
263
|
-
ciphertext hash
|
|
290
|
+
- `src/uploadCrypto.mjs` — filename encryption, the suite-0x03 part writer,
|
|
291
|
+
upload proof, multipart ciphertext hash
|
|
264
292
|
- `src/upload.mjs` — create session, encrypt, upload (direct + multipart),
|
|
265
293
|
finalize; the streaming chunk reader
|
|
266
294
|
- `src/sync.mjs` — `sf sync`: manifest, change detection, reconcile pass, watch
|
package/package.json
CHANGED
|
@@ -1,8 +1,8 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@shieldfive/cli",
|
|
3
|
-
"version": "0.3.
|
|
3
|
+
"version": "0.3.1",
|
|
4
4
|
"type": "module",
|
|
5
|
-
"description": "Open-source command-line client for ShieldFive: encrypt a folder on your machine and sync it to your EU-hosted
|
|
5
|
+
"description": "Open-source command-line client for ShieldFive: encrypt a folder on your machine and sync it to your EU-hosted vault with post-quantum hybrid encryption (ML-KEM-1024 + X25519). Read the code; it encrypts before upload.",
|
|
6
6
|
"license": "Apache-2.0",
|
|
7
7
|
"author": "Cho Garcia <security@shieldfive.com>",
|
|
8
8
|
"homepage": "https://github.com/shieldfive/cli",
|
|
@@ -46,8 +46,11 @@
|
|
|
46
46
|
"dependencies": {
|
|
47
47
|
"@noble/ciphers": "^2.1.1",
|
|
48
48
|
"@noble/hashes": "^2.0.1",
|
|
49
|
-
"@shieldfive/crypto": "
|
|
49
|
+
"@shieldfive/crypto": "~1.0.1",
|
|
50
50
|
"@supabase/supabase-js": "^2",
|
|
51
51
|
"libsodium-wrappers-sumo": "^0.8.2"
|
|
52
|
+
},
|
|
53
|
+
"optionalDependencies": {
|
|
54
|
+
"argon2": "0.45.1"
|
|
52
55
|
}
|
|
53
56
|
}
|
package/src/argon2.mjs
ADDED
|
@@ -0,0 +1,69 @@
|
|
|
1
|
+
import { hkdfSync } from 'node:crypto'
|
|
2
|
+
|
|
3
|
+
// Argon2id master-secret derivation, byte-identical to
|
|
4
|
+
// @shieldfive/crypto/kdf/argon2id deriveMasterSecret (the web's and the
|
|
5
|
+
// vault's reference implementation): libsodium presets, Argon2id v0x13,
|
|
6
|
+
// parallelism 1, 32-byte output, and salts longer than 16 bytes compressed
|
|
7
|
+
// through HKDF-SHA-256 (zero salt, info 'shieldfive/v1/argon2id/salt-compression').
|
|
8
|
+
//
|
|
9
|
+
// Two backends, same bytes:
|
|
10
|
+
//
|
|
11
|
+
// - native `argon2` (an optionalDependency): async, runs on the libuv thread
|
|
12
|
+
// pool so the event loop keeps turning, and several times faster.
|
|
13
|
+
// - @shieldfive/crypto over libsodium WASM: always installed. Used when the
|
|
14
|
+
// native module is missing, which is expected on platforms `argon2` ships no
|
|
15
|
+
// prebuilt binary for (Intel macOS, Windows on ARM) and where compiling it
|
|
16
|
+
// failed. npm drops a failed optional dependency instead of failing the
|
|
17
|
+
// install, so `npm i -g @shieldfive/cli` never hard-fails on it.
|
|
18
|
+
//
|
|
19
|
+
// Parameters are never reduced on either path. test/argon2.backends.test.mjs
|
|
20
|
+
// pins the two backends against each other and against web-produced vectors.
|
|
21
|
+
|
|
22
|
+
const PRESETS = {
|
|
23
|
+
moderate: { timeCost: 3, memoryCost: 262144 }, // OPSLIMIT/MEMLIMIT_MODERATE (256 MiB)
|
|
24
|
+
sensitive: { timeCost: 4, memoryCost: 1048576 }, // OPSLIMIT/MEMLIMIT_SENSITIVE (1 GiB)
|
|
25
|
+
}
|
|
26
|
+
|
|
27
|
+
let nativePromise
|
|
28
|
+
async function loadNative() {
|
|
29
|
+
if (process.env.SF_ARGON2_BACKEND === 'wasm') return null
|
|
30
|
+
nativePromise ??= import('argon2').then(
|
|
31
|
+
(m) => (typeof m.hash === 'function' ? m : m.default ?? null),
|
|
32
|
+
() => null,
|
|
33
|
+
)
|
|
34
|
+
return nativePromise
|
|
35
|
+
}
|
|
36
|
+
|
|
37
|
+
/** 'native' or 'wasm': which backend the next derivation will use. */
|
|
38
|
+
export async function argon2Backend() {
|
|
39
|
+
return (await loadNative()) ? 'native' : 'wasm'
|
|
40
|
+
}
|
|
41
|
+
|
|
42
|
+
function validate({ passphrase, salt, preset }) {
|
|
43
|
+
if (typeof passphrase !== 'string' || !passphrase) throw new Error('Passphrase is required')
|
|
44
|
+
if (!(salt instanceof Uint8Array) || salt.length < 16) throw new Error('Argon2id salt must be at least 16 bytes')
|
|
45
|
+
const params = Object.hasOwn(PRESETS, preset) ? PRESETS[preset] : null
|
|
46
|
+
if (!params) throw new Error(`Unsupported Argon2id preset: ${preset}`)
|
|
47
|
+
return params
|
|
48
|
+
}
|
|
49
|
+
|
|
50
|
+
export async function deriveMasterSecret({ passphrase, salt, preset = 'moderate' }) {
|
|
51
|
+
const params = validate({ passphrase, salt, preset })
|
|
52
|
+
const native = await loadNative()
|
|
53
|
+
if (!native) {
|
|
54
|
+
const { deriveMasterSecret: wasmDerive } = await import('@shieldfive/crypto/kdf/argon2id')
|
|
55
|
+
const { masterSecret } = await wasmDerive({ passphrase, salt, preset })
|
|
56
|
+
return { masterSecret, salt, preset }
|
|
57
|
+
}
|
|
58
|
+
const usableSalt = salt.length === 16 ? Buffer.from(salt) : Buffer.from(hkdfSync(
|
|
59
|
+
'sha256', salt, Buffer.alloc(32), 'shieldfive/v1/argon2id/salt-compression', 16,
|
|
60
|
+
))
|
|
61
|
+
const input = Buffer.from(passphrase, 'utf8')
|
|
62
|
+
try {
|
|
63
|
+
const masterSecret = await native.hash(input, {
|
|
64
|
+
...params, salt: usableSalt, type: native.argon2id, version: 0x13,
|
|
65
|
+
parallelism: 1, hashLength: 32, raw: true,
|
|
66
|
+
})
|
|
67
|
+
return { masterSecret, salt, preset } // a Buffer, i.e. a Uint8Array the caller zeroes
|
|
68
|
+
} finally { input.fill(0); usableSalt.fill(0) }
|
|
69
|
+
}
|
package/src/auth.mjs
CHANGED
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
// ShieldFive CLI — authentication.
|
|
2
2
|
//
|
|
3
|
-
// Signs in
|
|
3
|
+
// Signs in through ShieldFive's version-aware login route to obtain a Bearer
|
|
4
|
+
// session, then installs it in Supabase and elevates to AAL2 when
|
|
4
5
|
// the account has two-factor authentication (TOTP) enabled. The upload routes
|
|
5
6
|
// accept this token, and their server-side MFA gate requires an AAL2 token for
|
|
6
7
|
// enrolled accounts — so the step-up here is what makes `sf push` / `sf sync`
|
|
@@ -11,6 +12,7 @@
|
|
|
11
12
|
// via getAuthenticatorAssuranceLevel.
|
|
12
13
|
|
|
13
14
|
import { createClient } from '@supabase/supabase-js'
|
|
15
|
+
import { buildSignInCredentials } from './loginSecret.mjs'
|
|
14
16
|
|
|
15
17
|
// If the account requires an AAL2 step-up, run the TOTP challenge/verify flow
|
|
16
18
|
// and return the elevated access token. Returns null when no step-up is needed
|
|
@@ -77,24 +79,48 @@ export async function stepUpMfaIfRequired({ supabase, getTotpCode }) {
|
|
|
77
79
|
}
|
|
78
80
|
|
|
79
81
|
export async function signIn({
|
|
82
|
+
apiBaseUrl,
|
|
80
83
|
supabaseUrl,
|
|
81
84
|
anonKey,
|
|
82
85
|
email,
|
|
83
86
|
password,
|
|
84
87
|
getTotpCode,
|
|
88
|
+
fetchImpl = fetch,
|
|
89
|
+
createClientImpl = createClient,
|
|
85
90
|
}) {
|
|
86
|
-
|
|
91
|
+
// The raw password is never handed to Supabase directly. The salt lookup
|
|
92
|
+
// decides what the version-aware login route receives: only the derived
|
|
93
|
+
// login secret for a version-2 account; the password only for an account the
|
|
94
|
+
// server affirmatively reported as version 1. test/auth.noPasswordLeak.test.mjs
|
|
95
|
+
// pins this across every path.
|
|
96
|
+
email = String(email ?? '').trim().toLowerCase()
|
|
97
|
+
const credentials = await buildSignInCredentials({ apiBaseUrl, email, password, fetchImpl })
|
|
98
|
+
let response
|
|
99
|
+
try {
|
|
100
|
+
response = await fetchImpl(new URL('/api/mobile/auth/login', apiBaseUrl), {
|
|
101
|
+
method: 'POST', headers: { 'Content-Type': 'application/json', Accept: 'application/json' },
|
|
102
|
+
body: JSON.stringify({ email, ...credentials }), signal: AbortSignal.timeout(30_000),
|
|
103
|
+
})
|
|
104
|
+
} catch {
|
|
105
|
+
throw new Error('Sign-in failed: could not reach ShieldFive. Check your connection and try again.')
|
|
106
|
+
}
|
|
107
|
+
const payload = await response.json().catch(() => null)
|
|
108
|
+
if (!response.ok || typeof payload?.accessToken !== 'string' || !payload.accessToken ||
|
|
109
|
+
typeof payload?.refreshToken !== 'string' || !payload.refreshToken) {
|
|
110
|
+
throw new Error(response.status === 429 ? 'Too many sign-in attempts. Try again shortly.' : 'Sign-in failed. Check your credentials and try again.')
|
|
111
|
+
}
|
|
112
|
+
const supabase = createClientImpl(supabaseUrl, anonKey, {
|
|
87
113
|
auth: {
|
|
88
114
|
persistSession: false,
|
|
89
115
|
autoRefreshToken: false,
|
|
90
116
|
detectSessionInUrl: false,
|
|
91
117
|
},
|
|
92
118
|
})
|
|
93
|
-
const { data, error } = await supabase.auth.
|
|
94
|
-
|
|
95
|
-
|
|
119
|
+
const { data, error } = await supabase.auth.setSession({
|
|
120
|
+
access_token: payload.accessToken,
|
|
121
|
+
refresh_token: payload.refreshToken,
|
|
96
122
|
})
|
|
97
|
-
if (error) throw new Error(
|
|
123
|
+
if (error) throw new Error('Could not establish the sign-in session. Please try again.')
|
|
98
124
|
|
|
99
125
|
const aal1Token = data.session?.access_token
|
|
100
126
|
if (!aal1Token) {
|
package/src/cli.mjs
CHANGED
|
@@ -35,6 +35,7 @@ function usage() {
|
|
|
35
35
|
' [--interval=<seconds>] poll interval for --watch (default 5)',
|
|
36
36
|
' verify <file>... report whether each file is safely in your vault',
|
|
37
37
|
' encrypt <folder> encrypt each file locally, upload nothing',
|
|
38
|
+
' inspect <folder> count a migration source recursively; upload nothing',
|
|
38
39
|
' --help',
|
|
39
40
|
'',
|
|
40
41
|
'After sf login, push / sync / verify go through the agent and need nothing',
|
|
@@ -143,7 +144,8 @@ async function authAndUnlock(cfg) {
|
|
|
143
144
|
const { fetchAndUnlockVault } = await import('./vault.mjs')
|
|
144
145
|
|
|
145
146
|
process.stderr.write('signing in…\n')
|
|
146
|
-
const { accessToken } = await signIn({
|
|
147
|
+
const { accessToken, refreshToken, expiresAt } = await signIn({
|
|
148
|
+
apiBaseUrl: cfg.apiBaseUrl,
|
|
147
149
|
supabaseUrl: cfg.supabaseUrl,
|
|
148
150
|
anonKey: cfg.anonKey,
|
|
149
151
|
email: cfg.email,
|
|
@@ -157,7 +159,13 @@ async function authAndUnlock(cfg) {
|
|
|
157
159
|
accessToken,
|
|
158
160
|
password: cfg.vaultPassword,
|
|
159
161
|
})
|
|
160
|
-
|
|
162
|
+
// `sf sync --watch` outlives the access token (about an hour); it refreshes
|
|
163
|
+
// through this, the same exchange the agent uses.
|
|
164
|
+
const refreshSession = async (token) => {
|
|
165
|
+
const { refreshAccessToken } = await import('./auth.mjs')
|
|
166
|
+
return refreshAccessToken({ supabaseUrl: cfg.supabaseUrl, anonKey: cfg.anonKey, refreshToken: token })
|
|
167
|
+
}
|
|
168
|
+
return { accessToken, refreshToken, expiresAt, refreshSession, rootKey }
|
|
161
169
|
}
|
|
162
170
|
|
|
163
171
|
async function runningAgent() {
|
|
@@ -540,6 +548,12 @@ async function main() {
|
|
|
540
548
|
} else if (cmd === 'encrypt') {
|
|
541
549
|
if (!folder) throw new Error('encrypt needs a <folder>')
|
|
542
550
|
await cmdEncrypt(folder)
|
|
551
|
+
} else if (cmd === 'inspect') {
|
|
552
|
+
if (!folder) throw new Error('inspect needs a <folder>')
|
|
553
|
+
const { inspectMigrationSource } = await import('./migrationInventory.mjs')
|
|
554
|
+
const result = await inspectMigrationSource(folder)
|
|
555
|
+
process.stdout.write(JSON.stringify(result) + '\n')
|
|
556
|
+
if (result.skipped) process.exitCode = 2
|
|
543
557
|
} else if (cmd === 'push') {
|
|
544
558
|
if (!folder) throw new Error('push needs a <folder>')
|
|
545
559
|
await cmdPush(folder)
|
|
@@ -0,0 +1,63 @@
|
|
|
1
|
+
// Client-side derivation of the LOGIN SECRET, byte-identical to the web
|
|
2
|
+
// (utils/auth/loginSecret.ts) and Android (src/features/auth/loginSecret.ts):
|
|
3
|
+
//
|
|
4
|
+
// loginSecret = base64url_unpadded(
|
|
5
|
+
// Argon2id( DOMAIN || "\n" || password, authSalt, moderate ) )
|
|
6
|
+
//
|
|
7
|
+
// The password is used EXACTLY as typed: no trim, no Unicode normalisation.
|
|
8
|
+
// That is what web sign-in, signup, reset and change-password all feed this
|
|
9
|
+
// derivation, so it is what the secret GoTrue holds was derived from. (The
|
|
10
|
+
// vault KDF is different: the web keyring trims the vault password. See
|
|
11
|
+
// src/unlock.mjs.)
|
|
12
|
+
import { deriveMasterSecret } from './argon2.mjs'
|
|
13
|
+
|
|
14
|
+
const DOMAIN = 'shieldfive/v1/auth/login-secret'
|
|
15
|
+
|
|
16
|
+
export async function deriveLoginSecret({ password, authSalt }) {
|
|
17
|
+
if (typeof password !== 'string' || !password) throw new Error('Password is required')
|
|
18
|
+
if (typeof authSalt !== 'string' || !/^[A-Za-z0-9+/_-]+={0,2}$/.test(authSalt)) {
|
|
19
|
+
throw new Error('Invalid secure sign-in salt')
|
|
20
|
+
}
|
|
21
|
+
// Node's base64 decoder accepts base64 and base64url alike, as the web does.
|
|
22
|
+
const salt = new Uint8Array(Buffer.from(authSalt, 'base64'))
|
|
23
|
+
if (salt.length < 16 || salt.length > 64) throw new Error('Invalid secure sign-in salt')
|
|
24
|
+
const { masterSecret } = await deriveMasterSecret({
|
|
25
|
+
passphrase: `${DOMAIN}\n${password}`, salt, preset: 'moderate',
|
|
26
|
+
})
|
|
27
|
+
try { return Buffer.from(masterSecret).toString('base64url') }
|
|
28
|
+
finally { masterSecret.fill(0) }
|
|
29
|
+
}
|
|
30
|
+
|
|
31
|
+
/**
|
|
32
|
+
* The credential half of the sign-in body, per POST /api/auth/salt:
|
|
33
|
+
*
|
|
34
|
+
* version 2 -> { loginSecret } (no password)
|
|
35
|
+
* version 1, migration on -> { password, loginSecret, authSalt } (server verifies, then migrates)
|
|
36
|
+
* version 1, migration off -> { password }
|
|
37
|
+
*
|
|
38
|
+
* The raw password is only ever returned when the server AFFIRMATIVELY said
|
|
39
|
+
* version 1. Unlike the web's fetchAuthSalt, a failed, throttled or malformed
|
|
40
|
+
* lookup never falls back to version 1: for a migrated account that would post
|
|
41
|
+
* the password to the server and still not sign in. Throws before anything is
|
|
42
|
+
* posted to the login route.
|
|
43
|
+
*/
|
|
44
|
+
export async function buildSignInCredentials({ apiBaseUrl, email, password, fetchImpl = fetch }) {
|
|
45
|
+
let response
|
|
46
|
+
try {
|
|
47
|
+
response = await fetchImpl(new URL('/api/auth/salt', apiBaseUrl), {
|
|
48
|
+
method: 'POST', headers: { 'Content-Type': 'application/json', Accept: 'application/json' },
|
|
49
|
+
body: JSON.stringify({ email }), signal: AbortSignal.timeout(30_000),
|
|
50
|
+
})
|
|
51
|
+
} catch {
|
|
52
|
+
throw new Error('Unable to prepare secure sign-in. Check your connection and try again.')
|
|
53
|
+
}
|
|
54
|
+
if (response.status === 429) throw new Error('Too many sign-in attempts. Try again shortly.')
|
|
55
|
+
const body = await response.json().catch(() => null)
|
|
56
|
+
if (!response.ok || !body || ![1, 2].includes(body.authVersion) ||
|
|
57
|
+
typeof body.authSalt !== 'string' || !body.authSalt) {
|
|
58
|
+
throw new Error('Unable to prepare secure sign-in. Please try again.')
|
|
59
|
+
}
|
|
60
|
+
if (body.authVersion === 1 && body.migrationEnabled !== true) return { password }
|
|
61
|
+
const loginSecret = await deriveLoginSecret({ password, authSalt: body.authSalt })
|
|
62
|
+
return body.authVersion === 2 ? { loginSecret } : { password, loginSecret, authSalt: body.authSalt }
|
|
63
|
+
}
|
|
@@ -0,0 +1,45 @@
|
|
|
1
|
+
import { lstat, opendir } from 'node:fs/promises'
|
|
2
|
+
import { join, relative, resolve, sep } from 'node:path'
|
|
3
|
+
|
|
4
|
+
// Read-only preflight: includes hidden files and empty directories, preserves
|
|
5
|
+
// names, and reports unsupported entries instead of silently omitting them.
|
|
6
|
+
// It streams metadata without reading file contents or holding an inventory in
|
|
7
|
+
// memory. Upload must revalidate each source: this is not an immutable snapshot.
|
|
8
|
+
export async function* inventoryFolder(folder) {
|
|
9
|
+
const root = resolve(folder)
|
|
10
|
+
const initial = await lstat(root)
|
|
11
|
+
if (!initial.isDirectory() || initial.isSymbolicLink()) throw new Error('Select a real source folder, not a symbolic link')
|
|
12
|
+
const pending = [root]
|
|
13
|
+
while (pending.length) {
|
|
14
|
+
const directory = pending.pop()
|
|
15
|
+
const directoryInfo = await lstat(directory)
|
|
16
|
+
if (directoryInfo.isSymbolicLink() || !directoryInfo.isDirectory()) {
|
|
17
|
+
yield { kind: 'skipped', path: relative(root, directory).split(sep).join('/'), reason: 'source_changed' }
|
|
18
|
+
continue
|
|
19
|
+
}
|
|
20
|
+
for await (const entry of await opendir(directory)) {
|
|
21
|
+
const absolutePath = join(directory, entry.name)
|
|
22
|
+
const path = relative(root, absolutePath).split(sep).join('/')
|
|
23
|
+
let info
|
|
24
|
+
try { info = await lstat(absolutePath) }
|
|
25
|
+
catch (error) { yield { kind: 'skipped', path, reason: error.code ?? 'unreadable' }; continue }
|
|
26
|
+
if (info.isSymbolicLink()) yield { kind: 'skipped', path, reason: 'symbolic_link' }
|
|
27
|
+
else if (info.isDirectory()) {
|
|
28
|
+
yield { kind: 'directory', path }
|
|
29
|
+
pending.push(absolutePath)
|
|
30
|
+
} else if (info.isFile()) yield { kind: 'file', path, size: info.size, mtimeMs: info.mtimeMs }
|
|
31
|
+
else yield { kind: 'skipped', path, reason: 'unsupported_file_type' }
|
|
32
|
+
}
|
|
33
|
+
}
|
|
34
|
+
}
|
|
35
|
+
|
|
36
|
+
export async function inspectMigrationSource(folder) {
|
|
37
|
+
const totals = { files: 0, directories: 0, bytes: 0, skipped: 0 }
|
|
38
|
+
for await (const entry of inventoryFolder(folder)) {
|
|
39
|
+
if (entry.kind === 'file') { totals.files++; totals.bytes += entry.size }
|
|
40
|
+
else if (entry.kind === 'directory') totals.directories++
|
|
41
|
+
else totals.skipped++
|
|
42
|
+
if (!Number.isSafeInteger(totals.bytes)) throw new Error('Source size exceeds the supported inventory limit')
|
|
43
|
+
}
|
|
44
|
+
return totals
|
|
45
|
+
}
|
package/src/sync.mjs
CHANGED
|
@@ -11,9 +11,11 @@
|
|
|
11
11
|
// Scope notes: change is detected by (size, mtimeMs) — the rsync default, cheap
|
|
12
12
|
// and good enough; a same-size same-mtime edit is not re-uploaded. Deletions
|
|
13
13
|
// and renames are NOT mirrored (a file removed locally stays in the vault); sync
|
|
14
|
-
// is append-only for now. Each changed file becomes a new vault upload
|
|
15
|
-
// server does not overwrite by name)
|
|
16
|
-
//
|
|
14
|
+
// is append-only for now. Each changed file becomes a new vault upload at the
|
|
15
|
+
// vault root (the server does not overwrite by name); earlier versions are kept,
|
|
16
|
+
// not replaced or trashed, so they count against storage until removed in the
|
|
17
|
+
// app. Replacing the previous fileId needs the Bin's folder key to re-wrap it,
|
|
18
|
+
// which this client does not hold yet.
|
|
17
19
|
|
|
18
20
|
import { open, readFile, readdir, rename, stat } from 'node:fs/promises'
|
|
19
21
|
import { join } from 'node:path'
|
|
@@ -95,6 +97,11 @@ export function planSync(entries, manifest) {
|
|
|
95
97
|
// One reconcile pass. uploadFn is injected (defaults to the real uploadFile) so
|
|
96
98
|
// the pass is unit-testable without a network. Returns a summary and mutates
|
|
97
99
|
// `manifest` in place, persisting after each successful upload.
|
|
100
|
+
//
|
|
101
|
+
// `accessToken` is a string, or a function returning one; the function is
|
|
102
|
+
// called before every upload, so a long pass picks up a refreshed session. If
|
|
103
|
+
// it throws (the session cannot be refreshed) the pass stops and the error
|
|
104
|
+
// propagates: nothing after it could succeed.
|
|
98
105
|
export async function syncOnce({
|
|
99
106
|
apiBaseUrl,
|
|
100
107
|
accessToken,
|
|
@@ -112,11 +119,13 @@ export async function syncOnce({
|
|
|
112
119
|
let failed = 0
|
|
113
120
|
const errors = []
|
|
114
121
|
|
|
122
|
+
const tokenFor = typeof accessToken === 'function' ? accessToken : async () => accessToken
|
|
115
123
|
for (const entry of toUpload) {
|
|
124
|
+
const token = await tokenFor()
|
|
116
125
|
try {
|
|
117
126
|
const { fileId } = await uploadFn({
|
|
118
127
|
apiBaseUrl,
|
|
119
|
-
accessToken,
|
|
128
|
+
accessToken: token,
|
|
120
129
|
rootKey,
|
|
121
130
|
name: entry.name,
|
|
122
131
|
path: entry.path,
|
|
@@ -156,9 +165,47 @@ function sleep(ms, signal) {
|
|
|
156
165
|
})
|
|
157
166
|
}
|
|
158
167
|
|
|
168
|
+
// Refresh this long before the access token expires (seconds).
|
|
169
|
+
const REFRESH_MARGIN_S = 120
|
|
170
|
+
|
|
171
|
+
/**
|
|
172
|
+
* The access token for the next request: the current one while it has more
|
|
173
|
+
* than REFRESH_MARGIN_S left, otherwise a refreshed one. Supabase rotates
|
|
174
|
+
* refresh tokens and revokes the session if one is presented twice, so the
|
|
175
|
+
* refresh is single-flight and the rotated token replaces the old one.
|
|
176
|
+
* Without an expiry or a refresh function (tests, old callers) the token is
|
|
177
|
+
* used as is.
|
|
178
|
+
*/
|
|
179
|
+
export function createSessionTokens({ accessToken, refreshToken, expiresAt, refresh, now = () => Date.now() }) {
|
|
180
|
+
let tokens = { access: accessToken, refresh: refreshToken, expiresAt }
|
|
181
|
+
let inflight = null
|
|
182
|
+
return async function current() {
|
|
183
|
+
if (!refresh || !tokens.expiresAt || now() / 1000 < tokens.expiresAt - REFRESH_MARGIN_S) {
|
|
184
|
+
return tokens.access
|
|
185
|
+
}
|
|
186
|
+
inflight ??= refresh(tokens.refresh)
|
|
187
|
+
.then((s) => {
|
|
188
|
+
tokens = { access: s.accessToken, refresh: s.refreshToken, expiresAt: s.expiresAt }
|
|
189
|
+
return tokens.access
|
|
190
|
+
})
|
|
191
|
+
.finally(() => {
|
|
192
|
+
inflight = null
|
|
193
|
+
})
|
|
194
|
+
return inflight
|
|
195
|
+
}
|
|
196
|
+
}
|
|
197
|
+
|
|
159
198
|
// Orchestrates a real sync: sign in + unlock once, then run one pass (or loop on
|
|
160
199
|
// an interval under --watch). authAndUnlock is injected so the CLI can share its
|
|
161
|
-
// sign-in/unlock path and tests can stub it.
|
|
200
|
+
// sign-in/unlock path and tests can stub it. It may also return refreshToken,
|
|
201
|
+
// expiresAt (unix seconds) and refreshSession(refreshToken); with them the
|
|
202
|
+
// session is refreshed before it expires, so `--watch` keeps working past the
|
|
203
|
+
// access token's lifetime (about an hour). A refresh that fails ends the sync
|
|
204
|
+
// with an error (non-zero exit) instead of failing every file on every pass
|
|
205
|
+
// while looking healthy to a supervisor.
|
|
206
|
+
//
|
|
207
|
+
// Returns the last pass's summary. A single pass with failed files rejects, so
|
|
208
|
+
// `sf sync` exits non-zero; under --watch failed files are retried next pass.
|
|
162
209
|
export async function runSync({
|
|
163
210
|
cfg,
|
|
164
211
|
folder,
|
|
@@ -168,8 +215,17 @@ export async function runSync({
|
|
|
168
215
|
uploadFn,
|
|
169
216
|
log = (m) => process.stdout.write(`${m}\n`),
|
|
170
217
|
status = (m) => process.stderr.write(`${m}\n`),
|
|
218
|
+
now = () => Date.now(),
|
|
171
219
|
}) {
|
|
172
|
-
const
|
|
220
|
+
const session = await authAndUnlock(cfg)
|
|
221
|
+
const { rootKey } = session
|
|
222
|
+
const accessToken = createSessionTokens({
|
|
223
|
+
accessToken: session.accessToken,
|
|
224
|
+
refreshToken: session.refreshToken,
|
|
225
|
+
expiresAt: session.expiresAt,
|
|
226
|
+
refresh: session.refreshSession,
|
|
227
|
+
now,
|
|
228
|
+
})
|
|
173
229
|
const manifestPath = join(folder, MANIFEST_NAME)
|
|
174
230
|
const manifest = await loadManifest(manifestPath)
|
|
175
231
|
|
|
@@ -180,9 +236,10 @@ export async function runSync({
|
|
|
180
236
|
}
|
|
181
237
|
if (watch) process.on('SIGINT', onSigint)
|
|
182
238
|
|
|
239
|
+
let summary
|
|
183
240
|
try {
|
|
184
241
|
do {
|
|
185
|
-
|
|
242
|
+
summary = await syncOnce({
|
|
186
243
|
apiBaseUrl: cfg.apiBaseUrl,
|
|
187
244
|
accessToken,
|
|
188
245
|
rootKey,
|
|
@@ -202,4 +259,8 @@ export async function runSync({
|
|
|
202
259
|
} finally {
|
|
203
260
|
if (watch) process.off('SIGINT', onSigint)
|
|
204
261
|
}
|
|
262
|
+
if (!watch && summary?.failed) {
|
|
263
|
+
throw new Error(`${summary.failed} file(s) failed to sync.`)
|
|
264
|
+
}
|
|
265
|
+
return summary
|
|
205
266
|
}
|