@shieldfive/cli 0.3.1 → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +40 -0
- package/README.md +29 -11
- package/package.json +2 -2
- package/src/agent/server.mjs +4 -2
- package/src/auth.mjs +15 -1
- package/src/authPin.mjs +95 -0
- package/src/cli.mjs +63 -22
- package/src/loginSecret.mjs +10 -1
- package/src/sync.mjs +10 -5
- package/src/upload.mjs +162 -49
- package/src/uploadCrypto.mjs +26 -2
package/CHANGELOG.md
CHANGED
|
@@ -5,6 +5,46 @@ All notable changes to `@shieldfive/cli` will be documented in this file.
|
|
|
5
5
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
|
6
6
|
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
7
|
|
|
8
|
+
## Unreleased
|
|
9
|
+
|
|
10
|
+
## 0.4.0 — 2026-10-12
|
|
11
|
+
|
|
12
|
+
### Behaviour changes
|
|
13
|
+
|
|
14
|
+
- `sf push` now skips dotfiles (`.env`, `.git`, ...) by default; pass
|
|
15
|
+
`--include-dotfiles` to upload them as before.
|
|
16
|
+
- Sign-in refuses a server answer of auth version 1 for an account previously
|
|
17
|
+
signed in with version 2; pass `--allow-legacy-login` to override.
|
|
18
|
+
- Symlinks are skipped instead of followed.
|
|
19
|
+
|
|
20
|
+
### Security
|
|
21
|
+
|
|
22
|
+
- **Symlinks are not followed.** `push`, `sync` and the agent list files with
|
|
23
|
+
`lstat` and skip symlinks, and files are opened with `O_NOFOLLOW`, so a link
|
|
24
|
+
inside the folder can no longer upload a file from outside it.
|
|
25
|
+
- **`push` skips dotfiles by default** (`.env`, `.git`, ...), as `sync` already
|
|
26
|
+
did. `sf push --include-dotfiles` restores the old behaviour.
|
|
27
|
+
- **Sync manifest written 0600.** `.shieldfive-sync.json` lists file names and
|
|
28
|
+
vault ids; it is no longer world-readable.
|
|
29
|
+
- **README and npm description** now state the scheme precisely: contents are
|
|
30
|
+
AES-256-GCM under a post-quantum hybrid key, names are AES-256-GCM under the
|
|
31
|
+
vault key, and size, part hashes and timestamps are visible to the server.
|
|
32
|
+
|
|
33
|
+
### Fixed
|
|
34
|
+
|
|
35
|
+
- A failed upload cancels its pending session (`POST /api/files/delete`, as the
|
|
36
|
+
web app does) instead of leaving a row holding quota until the reaper runs.
|
|
37
|
+
Not done when only the finalize answer was lost, since the file may be in.
|
|
38
|
+
- 0-byte files: uploaded as a header-only cipher-v3 object with the server's
|
|
39
|
+
empty-file proof when `/api/files/upload-capabilities` says `emptyCipherV3`;
|
|
40
|
+
otherwise skipped with a clear message (exit code 2) before any session.
|
|
41
|
+
- New uploads send a v6 row-bound encrypted name from upload start (#14).
|
|
42
|
+
- **Downgrade pin for sign-in.** After a sign-in with a login secret (auth
|
|
43
|
+
version 2), the CLI records the account (a SHA-256 of the email) in
|
|
44
|
+
`~/.shieldfive/auth-pins.json` (mode 0600). A later server answer of
|
|
45
|
+
version 1 for that account is refused before the raw password is sent.
|
|
46
|
+
`--allow-legacy-login` on `sf login`, `sf push` or `sf sync` overrides it.
|
|
47
|
+
|
|
8
48
|
## 0.3.1 - 2026-10-01
|
|
9
49
|
|
|
10
50
|
### Security
|
package/README.md
CHANGED
|
@@ -1,24 +1,35 @@
|
|
|
1
1
|
# @shieldfive/cli
|
|
2
2
|
|
|
3
3
|
> Command-line client for [ShieldFive](https://shieldfive.com). Encrypt a folder
|
|
4
|
-
> on your machine, then push or continuously sync it to your EU-hosted
|
|
5
|
-
>
|
|
6
|
-
>
|
|
4
|
+
> on your machine, then push or continuously sync it to your EU-hosted vault.
|
|
5
|
+
> File contents and names are encrypted locally before they leave your machine
|
|
6
|
+
> (contents with a post-quantum hybrid key). Read the code and verify that for
|
|
7
|
+
> yourself.
|
|
7
8
|
|
|
8
9
|
[](https://github.com/shieldfive/cli/actions/workflows/ci.yml)
|
|
9
10
|
[](LICENSE)
|
|
10
11
|
[](#honest-scope)
|
|
11
12
|
|
|
12
13
|
The point of this tool being open source is that you do not have to trust a
|
|
13
|
-
marketing claim. The plaintext of your files is never sent to a
|
|
14
|
-
server
|
|
15
|
-
|
|
16
|
-
|
|
14
|
+
marketing claim. The plaintext of your files and their names is never sent to a
|
|
15
|
+
ShieldFive server:
|
|
16
|
+
|
|
17
|
+
- **File contents** — AES-256-GCM chunks under a per-file key that combines a
|
|
18
|
+
random envelope key (wrapped by your vault root key) with an ML-KEM-1024 +
|
|
19
|
+
X25519 hybrid encapsulation to your vault identity.
|
|
20
|
+
- **File names** — AES-256-GCM under a key derived from your vault root key
|
|
21
|
+
(not the post-quantum layer), plus a keyed HMAC of the lowercased name so the
|
|
22
|
+
server can spot duplicates without reading it.
|
|
23
|
+
|
|
24
|
+
What the server does see, by design: the plaintext size of each file, part
|
|
25
|
+
hashes (SHA-1 of ciphertext), the wrapped keys, an upload proof, timestamps and
|
|
26
|
+
your account. Sign-in sends a secret derived from your password, never the
|
|
27
|
+
password itself. Two files carry the whole story:
|
|
17
28
|
|
|
18
29
|
- [`src/uploadCrypto.mjs`](src/uploadCrypto.mjs) — the encryption applied to
|
|
19
30
|
every chunk and to the filename.
|
|
20
|
-
- [`src/upload.mjs`](src/upload.mjs) — the only place bytes are sent over
|
|
21
|
-
network. Every
|
|
31
|
+
- [`src/upload.mjs`](src/upload.mjs) — the only place file bytes are sent over
|
|
32
|
+
the network. Every file body it sends is ciphertext.
|
|
22
33
|
|
|
23
34
|
If you want to see encryption happen without an account at all, run
|
|
24
35
|
`sf encrypt <folder>` — it encrypts each file locally and prints the ciphertext
|
|
@@ -81,7 +92,7 @@ npm test # 107 tests
|
|
|
81
92
|
sf login [--idle=<hours>] sign in once; an agent holds the session in memory (locks after 8 h idle)
|
|
82
93
|
sf status is an agent running, and for which account
|
|
83
94
|
sf logout [--everywhere] revoke the session and stop the agent; --everywhere signs out every device
|
|
84
|
-
sf push <folder>
|
|
95
|
+
sf push <folder> [--include-dotfiles] encrypt and upload every file once (skips dotfiles unless asked; never follows symlinks)
|
|
85
96
|
sf sync <folder> [--watch] [--interval=N] upload new/changed files; --watch keeps a poll loop (N seconds, default 5)
|
|
86
97
|
sf verify <file>... report whether each file is safely stored in your vault
|
|
87
98
|
sf encrypt <folder> encrypt each file locally, upload nothing (no account needed)
|
|
@@ -89,7 +100,14 @@ sf inspect <folder> read-only recursive migration inven
|
|
|
89
100
|
```
|
|
90
101
|
|
|
91
102
|
`sf push` and `sf sync` pick the direct or multipart upload path automatically
|
|
92
|
-
from the file size.
|
|
103
|
+
from the file size. Both read only regular files directly inside the folder:
|
|
104
|
+
symlinks are skipped (and files are opened with `O_NOFOLLOW`), and names
|
|
105
|
+
starting with `.` are skipped unless you pass `sf push --include-dotfiles`.
|
|
106
|
+
An empty (0-byte) file is uploaded only when the server says it accepts empty
|
|
107
|
+
files; otherwise it is reported as skipped and the exit code is 2. If an upload
|
|
108
|
+
fails after its session was created, the pending session is cancelled so it
|
|
109
|
+
does not hold quota. `sf sync` writes its manifest (`.shieldfive-sync.json`)
|
|
110
|
+
with mode 0600.
|
|
93
111
|
|
|
94
112
|
`sf inspect` reports JSON counts of files, directories, bytes and skipped entries.
|
|
95
113
|
It includes nested/hidden files and empty folders, reads metadata only, and
|
package/package.json
CHANGED
|
@@ -1,8 +1,8 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@shieldfive/cli",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.4.0",
|
|
4
4
|
"type": "module",
|
|
5
|
-
"description": "Open-source command-line client for ShieldFive:
|
|
5
|
+
"description": "Open-source command-line client for ShieldFive: encrypts files on your machine before upload (contents: AES-256-GCM under an ML-KEM-1024 + X25519 hybrid key; names: AES-256-GCM under your vault key) and pushes or syncs them to your vault. Read the code.",
|
|
6
6
|
"license": "Apache-2.0",
|
|
7
7
|
"author": "Cho Garcia <security@shieldfive.com>",
|
|
8
8
|
"homepage": "https://github.com/shieldfive/cli",
|
package/src/agent/server.mjs
CHANGED
|
@@ -5,7 +5,7 @@
|
|
|
5
5
|
// See docs/agent-design.md for why it exists and what it will never do.
|
|
6
6
|
|
|
7
7
|
import { createHmac, randomUUID } from 'node:crypto'
|
|
8
|
-
import { chmod, open, readFile, rm, stat } from 'node:fs/promises'
|
|
8
|
+
import { chmod, lstat, open, readFile, rm, stat } from 'node:fs/promises'
|
|
9
9
|
import { connect, createServer } from 'node:net'
|
|
10
10
|
import { basename, dirname, join } from 'node:path'
|
|
11
11
|
|
|
@@ -189,7 +189,9 @@ export function createAgent({
|
|
|
189
189
|
}
|
|
190
190
|
|
|
191
191
|
async function upload(path) {
|
|
192
|
-
|
|
192
|
+
// lstat: a symlink is refused, not followed (and readFileChunks opens with
|
|
193
|
+
// O_NOFOLLOW, so a swap after this check fails too).
|
|
194
|
+
const before = await lstat(path)
|
|
193
195
|
if (!before.isFile()) throw new AgentError('not_a_file', `${path} is not a regular file.`)
|
|
194
196
|
const token = await accessToken()
|
|
195
197
|
|
package/src/auth.mjs
CHANGED
|
@@ -87,6 +87,10 @@ export async function signIn({
|
|
|
87
87
|
getTotpCode,
|
|
88
88
|
fetchImpl = fetch,
|
|
89
89
|
createClientImpl = createClient,
|
|
90
|
+
// { isPinned(email), pin(email) } -- src/authPin.mjs fileAuthPin(). Null
|
|
91
|
+
// means no pin (tests, embedders); the sf commands always pass one.
|
|
92
|
+
authPin = null,
|
|
93
|
+
allowLegacyLogin = false,
|
|
90
94
|
}) {
|
|
91
95
|
// The raw password is never handed to Supabase directly. The salt lookup
|
|
92
96
|
// decides what the version-aware login route receives: only the derived
|
|
@@ -94,7 +98,10 @@ export async function signIn({
|
|
|
94
98
|
// server affirmatively reported as version 1. test/auth.noPasswordLeak.test.mjs
|
|
95
99
|
// pins this across every path.
|
|
96
100
|
email = String(email ?? '').trim().toLowerCase()
|
|
97
|
-
const
|
|
101
|
+
const pinnedV2 = authPin ? await authPin.isPinned(email) : false
|
|
102
|
+
const credentials = await buildSignInCredentials({
|
|
103
|
+
apiBaseUrl, email, password, fetchImpl, pinnedV2, allowLegacyLogin,
|
|
104
|
+
})
|
|
98
105
|
let response
|
|
99
106
|
try {
|
|
100
107
|
response = await fetchImpl(new URL('/api/mobile/auth/login', apiBaseUrl), {
|
|
@@ -127,6 +134,13 @@ export async function signIn({
|
|
|
127
134
|
throw new Error('Sign-in returned no access token.')
|
|
128
135
|
}
|
|
129
136
|
|
|
137
|
+
// Signed in with only the login secret: the account is version 2. Remember
|
|
138
|
+
// it so a later version-1 answer is refused (src/authPin.mjs). A failed
|
|
139
|
+
// write does not fail the sign-in; the next one retries it.
|
|
140
|
+
if (authPin && !('password' in credentials)) {
|
|
141
|
+
try { await authPin.pin(email) } catch { /* retried on the next sign-in */ }
|
|
142
|
+
}
|
|
143
|
+
|
|
130
144
|
const aal2Token = await stepUpMfaIfRequired({ supabase, getTotpCode })
|
|
131
145
|
|
|
132
146
|
// The refresh token has to come from the session that was stepped up.
|
package/src/authPin.mjs
ADDED
|
@@ -0,0 +1,95 @@
|
|
|
1
|
+
// "This account signs in with a login secret" -- remembered on this computer.
|
|
2
|
+
//
|
|
3
|
+
// /api/auth/salt is the server telling the CLI how to sign in: version 2 means
|
|
4
|
+
// send only the derived login secret, version 1 means send the raw password.
|
|
5
|
+
// A server that has been tampered with (or an account a stolen session
|
|
6
|
+
// downgraded) can answer version 1 for an account that is really version 2,
|
|
7
|
+
// and a CLI that obeys sends the raw password -- the one thing version 2
|
|
8
|
+
// promises never leaves this machine. The mobile app and the web already pin;
|
|
9
|
+
// this is the CLI's pin.
|
|
10
|
+
//
|
|
11
|
+
// Once a sign-in has gone through with the server saying version 2, the
|
|
12
|
+
// account is recorded here. Afterwards a version-1 answer for it is refused
|
|
13
|
+
// unless the user passes --allow-legacy-login.
|
|
14
|
+
//
|
|
15
|
+
// File: ~/.shieldfive/auth-pins.json, mode 0600 in a 0700 directory. It holds
|
|
16
|
+
// SHA-256 hashes of lower-cased email addresses, not the addresses, so the file
|
|
17
|
+
// does not list which accounts this computer uses.
|
|
18
|
+
|
|
19
|
+
import { createHash } from 'node:crypto'
|
|
20
|
+
import { chmod, readFile, rename, writeFile } from 'node:fs/promises'
|
|
21
|
+
import { homedir } from 'node:os'
|
|
22
|
+
import { join } from 'node:path'
|
|
23
|
+
|
|
24
|
+
import { ensurePrivateDir, shieldfiveHome } from './agent/paths.mjs'
|
|
25
|
+
|
|
26
|
+
export const AUTH_PIN_FILE = 'auth-pins.json'
|
|
27
|
+
|
|
28
|
+
export class LegacyLoginRefusedError extends Error {
|
|
29
|
+
constructor() {
|
|
30
|
+
super(
|
|
31
|
+
'Refusing to sign in: this computer has signed in to this account with a ' +
|
|
32
|
+
'login secret before, but the server now asks for the raw password ' +
|
|
33
|
+
'(auth version 1). That should not happen to an account that has moved ' +
|
|
34
|
+
'to version 2, so the password was not sent. If you know why (for ' +
|
|
35
|
+
'example ShieldFive support reset the account), run the command again ' +
|
|
36
|
+
'with --allow-legacy-login.',
|
|
37
|
+
)
|
|
38
|
+
this.name = 'LegacyLoginRefusedError'
|
|
39
|
+
this.code = 'legacy_login_refused'
|
|
40
|
+
}
|
|
41
|
+
}
|
|
42
|
+
|
|
43
|
+
function emailKey(email) {
|
|
44
|
+
return createHash('sha256')
|
|
45
|
+
.update(String(email ?? '').trim().toLowerCase())
|
|
46
|
+
.digest('hex')
|
|
47
|
+
}
|
|
48
|
+
|
|
49
|
+
/**
|
|
50
|
+
* The pin store backed by ~/.shieldfive/auth-pins.json. `home` is injectable
|
|
51
|
+
* for tests. A file that cannot be read as a pin list is treated as holding
|
|
52
|
+
* no pins for reads, and is replaced on the next write.
|
|
53
|
+
*/
|
|
54
|
+
export function fileAuthPin({ home = homedir() } = {}) {
|
|
55
|
+
const dir = shieldfiveHome(home)
|
|
56
|
+
const file = join(dir, AUTH_PIN_FILE)
|
|
57
|
+
|
|
58
|
+
const readKeys = async () => {
|
|
59
|
+
let raw
|
|
60
|
+
try {
|
|
61
|
+
raw = await readFile(file, 'utf8')
|
|
62
|
+
} catch (err) {
|
|
63
|
+
if (err.code === 'ENOENT') return []
|
|
64
|
+
throw err
|
|
65
|
+
}
|
|
66
|
+
try {
|
|
67
|
+
const parsed = JSON.parse(raw)
|
|
68
|
+
return Array.isArray(parsed?.v2)
|
|
69
|
+
? parsed.v2.filter((k) => typeof k === 'string' && /^[0-9a-f]{64}$/.test(k))
|
|
70
|
+
: []
|
|
71
|
+
} catch {
|
|
72
|
+
return []
|
|
73
|
+
}
|
|
74
|
+
}
|
|
75
|
+
|
|
76
|
+
return {
|
|
77
|
+
file,
|
|
78
|
+
async isPinned(email) {
|
|
79
|
+
return (await readKeys()).includes(emailKey(email))
|
|
80
|
+
},
|
|
81
|
+
async pin(email) {
|
|
82
|
+
await ensurePrivateDir(dir)
|
|
83
|
+
const keys = await readKeys()
|
|
84
|
+
const key = emailKey(email)
|
|
85
|
+
if (keys.includes(key)) return
|
|
86
|
+
keys.push(key)
|
|
87
|
+
// Write-then-rename so a crash never leaves a half-written file, and
|
|
88
|
+
// chmod after rename in case an older file had looser bits.
|
|
89
|
+
const tmp = `${file}.${process.pid}.tmp`
|
|
90
|
+
await writeFile(tmp, JSON.stringify({ v2: keys }) + '\n', { mode: 0o600 })
|
|
91
|
+
await rename(tmp, file)
|
|
92
|
+
await chmod(file, 0o600)
|
|
93
|
+
},
|
|
94
|
+
}
|
|
95
|
+
}
|
package/src/cli.mjs
CHANGED
|
@@ -15,7 +15,7 @@
|
|
|
15
15
|
// signed in as another. See docs/agent-design.md.
|
|
16
16
|
|
|
17
17
|
import { webcrypto } from 'node:crypto'
|
|
18
|
-
import { readFile, readdir
|
|
18
|
+
import { lstat, readFile, readdir } from 'node:fs/promises'
|
|
19
19
|
import { join, resolve } from 'node:path'
|
|
20
20
|
|
|
21
21
|
import { promptHidden, promptLine } from './prompt.mjs'
|
|
@@ -31,6 +31,8 @@ function usage() {
|
|
|
31
31
|
' logout [--everywhere] revoke the session and stop the agent',
|
|
32
32
|
' --everywhere also signs out your browser and phone',
|
|
33
33
|
' push <folder> encrypt and upload every file in <folder>',
|
|
34
|
+
' [--include-dotfiles] also upload names starting with "."',
|
|
35
|
+
' (symlinks are never followed)',
|
|
34
36
|
' sync <folder> [--watch] upload new and changed files',
|
|
35
37
|
' [--interval=<seconds>] poll interval for --watch (default 5)',
|
|
36
38
|
' verify <file>... report whether each file is safely in your vault',
|
|
@@ -38,6 +40,10 @@ function usage() {
|
|
|
38
40
|
' inspect <folder> count a migration source recursively; upload nothing',
|
|
39
41
|
' --help',
|
|
40
42
|
'',
|
|
43
|
+
'login, push and sync accept --allow-legacy-login: sign in even though the',
|
|
44
|
+
'server asks for the raw password (auth version 1) for an account this',
|
|
45
|
+
'computer has signed in to with a login secret. Refused by default.',
|
|
46
|
+
'',
|
|
41
47
|
'After sf login, push / sync / verify go through the agent and need nothing',
|
|
42
48
|
'in the environment. Without an agent, or when SF_EMAIL and SF_PASSWORD are',
|
|
43
49
|
'set, push and sync sign in from those [SF_VAULT_PASSWORD if your vault',
|
|
@@ -48,10 +54,15 @@ function usage() {
|
|
|
48
54
|
)
|
|
49
55
|
}
|
|
50
56
|
|
|
51
|
-
|
|
57
|
+
// Regular files directly in `folder`. lstat, not stat: a symlink is skipped
|
|
58
|
+
// rather than followed, so `sf push ~/project` cannot upload ~/.ssh/id_ed25519
|
|
59
|
+
// through a link that points out of the folder. Dotfiles (.env, .git, .DS_Store,
|
|
60
|
+
// sync's own manifest) are skipped unless includeDotfiles is set, matching sync.
|
|
61
|
+
async function* filesIn(folder, { includeDotfiles = false } = {}) {
|
|
52
62
|
for (const name of await readdir(folder)) {
|
|
63
|
+
if (!includeDotfiles && name.startsWith('.')) continue
|
|
53
64
|
const path = join(folder, name)
|
|
54
|
-
const info = await
|
|
65
|
+
const info = await lstat(path)
|
|
55
66
|
if (info.isFile()) yield { name, path, size: info.size }
|
|
56
67
|
}
|
|
57
68
|
}
|
|
@@ -99,7 +110,7 @@ function envCredentialsSet() {
|
|
|
99
110
|
return Boolean(process.env.SF_EMAIL && process.env.SF_PASSWORD)
|
|
100
111
|
}
|
|
101
112
|
|
|
102
|
-
function readLiveConfig() {
|
|
113
|
+
function readLiveConfig(flags = {}) {
|
|
103
114
|
const e = process.env
|
|
104
115
|
const email = e.SF_EMAIL
|
|
105
116
|
const password = e.SF_PASSWORD
|
|
@@ -119,6 +130,7 @@ function readLiveConfig() {
|
|
|
119
130
|
email,
|
|
120
131
|
password,
|
|
121
132
|
vaultPassword: e.SF_VAULT_PASSWORD || password,
|
|
133
|
+
allowLegacyLogin: flags['allow-legacy-login'] === true,
|
|
122
134
|
}
|
|
123
135
|
}
|
|
124
136
|
|
|
@@ -141,6 +153,7 @@ async function readTotpCode() {
|
|
|
141
153
|
// lazily so `sf encrypt` runs without @supabase/supabase-js.
|
|
142
154
|
async function authAndUnlock(cfg) {
|
|
143
155
|
const { signIn } = await import('./auth.mjs')
|
|
156
|
+
const { fileAuthPin } = await import('./authPin.mjs')
|
|
144
157
|
const { fetchAndUnlockVault } = await import('./vault.mjs')
|
|
145
158
|
|
|
146
159
|
process.stderr.write('signing in…\n')
|
|
@@ -151,6 +164,8 @@ async function authAndUnlock(cfg) {
|
|
|
151
164
|
email: cfg.email,
|
|
152
165
|
password: cfg.password,
|
|
153
166
|
getTotpCode: readTotpCode,
|
|
167
|
+
authPin: fileAuthPin(),
|
|
168
|
+
allowLegacyLogin: cfg.allowLegacyLogin === true,
|
|
154
169
|
})
|
|
155
170
|
|
|
156
171
|
process.stderr.write('unlocking vault…\n')
|
|
@@ -298,10 +313,15 @@ async function cmdLogin(flags) {
|
|
|
298
313
|
(await promptHidden('Vault password (press Enter if it is the same): ')) || password
|
|
299
314
|
|
|
300
315
|
const { signIn } = await import('./auth.mjs')
|
|
316
|
+
const { fileAuthPin } = await import('./authPin.mjs')
|
|
301
317
|
const { fetchAndUnlockVault } = await import('./vault.mjs')
|
|
302
318
|
|
|
303
319
|
process.stderr.write('signing in…\n')
|
|
304
|
-
const session = await signIn({
|
|
320
|
+
const session = await signIn({
|
|
321
|
+
...cfg, email, password, getTotpCode: readTotpCode,
|
|
322
|
+
authPin: fileAuthPin(),
|
|
323
|
+
allowLegacyLogin: flags['allow-legacy-login'] === true,
|
|
324
|
+
})
|
|
305
325
|
|
|
306
326
|
process.stderr.write('unlocking vault…\n')
|
|
307
327
|
const rootKey = await fetchAndUnlockVault({
|
|
@@ -378,14 +398,22 @@ async function cmdLogout(flags) {
|
|
|
378
398
|
}
|
|
379
399
|
|
|
380
400
|
// login -> unlock -> encrypt -> upload every file in the folder, once.
|
|
381
|
-
async function cmdPush(folder) {
|
|
401
|
+
async function cmdPush(folder, flags) {
|
|
382
402
|
const { agent } = await chooseAuth()
|
|
383
403
|
if (agent) {
|
|
384
404
|
const { request } = await import('./agent/client.mjs')
|
|
385
405
|
process.stderr.write(`pushing to account ${agent.account} through the agent\n`)
|
|
386
406
|
let differs = 0
|
|
387
|
-
for await (const { name, path } of filesIn(resolve(folder))) {
|
|
388
|
-
|
|
407
|
+
for await (const { name, path } of filesIn(resolve(folder), pushListOptions(flags))) {
|
|
408
|
+
let result
|
|
409
|
+
try {
|
|
410
|
+
result = await request('upload', { path }, { timeoutMs: 0 })
|
|
411
|
+
} catch (err) {
|
|
412
|
+
if (err.code !== 'empty_file_unsupported') throw err
|
|
413
|
+
process.stdout.write(`skipped ${name}: ${err.message}\n`)
|
|
414
|
+
differs++
|
|
415
|
+
continue
|
|
416
|
+
}
|
|
389
417
|
if (result.matchesFileNow) {
|
|
390
418
|
process.stdout.write(`pushed ${name}\n`)
|
|
391
419
|
} else {
|
|
@@ -402,21 +430,34 @@ async function cmdPush(folder) {
|
|
|
402
430
|
return
|
|
403
431
|
}
|
|
404
432
|
|
|
405
|
-
const cfg = readLiveConfig()
|
|
433
|
+
const cfg = readLiveConfig(flags)
|
|
406
434
|
const { uploadFile } = await import('./upload.mjs')
|
|
407
435
|
const { accessToken, rootKey } = await authAndUnlock(cfg)
|
|
408
436
|
|
|
409
|
-
|
|
410
|
-
|
|
411
|
-
|
|
412
|
-
|
|
413
|
-
|
|
414
|
-
|
|
415
|
-
|
|
416
|
-
|
|
417
|
-
|
|
437
|
+
let skipped = 0
|
|
438
|
+
for await (const { name, path, size } of filesIn(folder, pushListOptions(flags))) {
|
|
439
|
+
try {
|
|
440
|
+
await uploadFile({
|
|
441
|
+
apiBaseUrl: cfg.apiBaseUrl,
|
|
442
|
+
accessToken,
|
|
443
|
+
rootKey,
|
|
444
|
+
name,
|
|
445
|
+
path,
|
|
446
|
+
size,
|
|
447
|
+
})
|
|
448
|
+
} catch (err) {
|
|
449
|
+
if (err.code !== 'empty_file_unsupported') throw err
|
|
450
|
+
process.stdout.write(`skipped ${name}: ${err.message}\n`)
|
|
451
|
+
skipped++
|
|
452
|
+
continue
|
|
453
|
+
}
|
|
418
454
|
process.stdout.write(`pushed ${name}\n`)
|
|
419
455
|
}
|
|
456
|
+
if (skipped) process.exitCode = 2
|
|
457
|
+
}
|
|
458
|
+
|
|
459
|
+
function pushListOptions(flags) {
|
|
460
|
+
return { includeDotfiles: flags['include-dotfiles'] === true }
|
|
420
461
|
}
|
|
421
462
|
|
|
422
463
|
async function syncViaAgent(folder, { watch, intervalMs }, agent) {
|
|
@@ -466,11 +507,11 @@ async function syncViaAgent(folder, { watch, intervalMs }, agent) {
|
|
|
466
507
|
}
|
|
467
508
|
|
|
468
509
|
// login -> unlock -> mirror new/changed files into the vault, once or on a loop.
|
|
469
|
-
async function cmdSync(folder, { watch, intervalMs }) {
|
|
510
|
+
async function cmdSync(folder, { watch, intervalMs }, flags) {
|
|
470
511
|
const { agent } = await chooseAuth()
|
|
471
512
|
if (agent) return syncViaAgent(resolve(folder), { watch, intervalMs }, agent)
|
|
472
513
|
|
|
473
|
-
const cfg = readLiveConfig()
|
|
514
|
+
const cfg = readLiveConfig(flags)
|
|
474
515
|
const { uploadFile } = await import('./upload.mjs')
|
|
475
516
|
const { runSync } = await import('./sync.mjs')
|
|
476
517
|
|
|
@@ -556,7 +597,7 @@ async function main() {
|
|
|
556
597
|
if (result.skipped) process.exitCode = 2
|
|
557
598
|
} else if (cmd === 'push') {
|
|
558
599
|
if (!folder) throw new Error('push needs a <folder>')
|
|
559
|
-
await cmdPush(folder)
|
|
600
|
+
await cmdPush(folder, flags)
|
|
560
601
|
} else if (cmd === 'sync') {
|
|
561
602
|
if (!folder) throw new Error('sync needs a <folder>')
|
|
562
603
|
// `--interval 10` (space-separated) would otherwise parse as a bare flag and
|
|
@@ -569,7 +610,7 @@ async function main() {
|
|
|
569
610
|
await cmdSync(folder, {
|
|
570
611
|
watch: Boolean(flags.watch),
|
|
571
612
|
intervalMs: Math.round(intervalSec * 1000),
|
|
572
|
-
})
|
|
613
|
+
}, flags)
|
|
573
614
|
} else if (cmd === 'verify') {
|
|
574
615
|
await cmdVerify(positionals)
|
|
575
616
|
} else {
|
package/src/loginSecret.mjs
CHANGED
|
@@ -10,6 +10,7 @@
|
|
|
10
10
|
// vault KDF is different: the web keyring trims the vault password. See
|
|
11
11
|
// src/unlock.mjs.)
|
|
12
12
|
import { deriveMasterSecret } from './argon2.mjs'
|
|
13
|
+
import { LegacyLoginRefusedError } from './authPin.mjs'
|
|
13
14
|
|
|
14
15
|
const DOMAIN = 'shieldfive/v1/auth/login-secret'
|
|
15
16
|
|
|
@@ -40,8 +41,15 @@ export async function deriveLoginSecret({ password, authSalt }) {
|
|
|
40
41
|
* lookup never falls back to version 1: for a migrated account that would post
|
|
41
42
|
* the password to the server and still not sign in. Throws before anything is
|
|
42
43
|
* posted to the login route.
|
|
44
|
+
*
|
|
45
|
+
* `pinnedV2`: this computer has already signed in to the account as version 2
|
|
46
|
+
* (src/authPin.mjs). A version-1 answer for it is then refused with
|
|
47
|
+
* LegacyLoginRefusedError, before the password goes anywhere, unless
|
|
48
|
+
* `allowLegacyLogin` (the --allow-legacy-login flag) says the user expects it.
|
|
43
49
|
*/
|
|
44
|
-
export async function buildSignInCredentials({
|
|
50
|
+
export async function buildSignInCredentials({
|
|
51
|
+
apiBaseUrl, email, password, fetchImpl = fetch, pinnedV2 = false, allowLegacyLogin = false,
|
|
52
|
+
}) {
|
|
45
53
|
let response
|
|
46
54
|
try {
|
|
47
55
|
response = await fetchImpl(new URL('/api/auth/salt', apiBaseUrl), {
|
|
@@ -57,6 +65,7 @@ export async function buildSignInCredentials({ apiBaseUrl, email, password, fetc
|
|
|
57
65
|
typeof body.authSalt !== 'string' || !body.authSalt) {
|
|
58
66
|
throw new Error('Unable to prepare secure sign-in. Please try again.')
|
|
59
67
|
}
|
|
68
|
+
if (body.authVersion === 1 && pinnedV2 && !allowLegacyLogin) throw new LegacyLoginRefusedError()
|
|
60
69
|
if (body.authVersion === 1 && body.migrationEnabled !== true) return { password }
|
|
61
70
|
const loginSecret = await deriveLoginSecret({ password, authSalt: body.authSalt })
|
|
62
71
|
return body.authVersion === 2 ? { loginSecret } : { password, loginSecret, authSalt: body.authSalt }
|
package/src/sync.mjs
CHANGED
|
@@ -17,7 +17,7 @@
|
|
|
17
17
|
// app. Replacing the previous fileId needs the Bin's folder key to re-wrap it,
|
|
18
18
|
// which this client does not hold yet.
|
|
19
19
|
|
|
20
|
-
import { open, readFile, readdir, rename
|
|
20
|
+
import { lstat, open, readFile, readdir, rename } from 'node:fs/promises'
|
|
21
21
|
import { join } from 'node:path'
|
|
22
22
|
|
|
23
23
|
export const MANIFEST_NAME = '.shieldfive-sync.json'
|
|
@@ -28,7 +28,8 @@ export function signatureOf({ size, mtimeMs }) {
|
|
|
28
28
|
return `${size}:${mtimeMs}`
|
|
29
29
|
}
|
|
30
30
|
|
|
31
|
-
// List regular files to consider for sync: skips directories,
|
|
31
|
+
// List regular files to consider for sync: skips directories, symlinks (lstat,
|
|
32
|
+
// so a link never pulls in a file from outside the folder), dotfiles (which
|
|
32
33
|
// includes the manifest itself), and any explicitly excluded names.
|
|
33
34
|
export async function scanFiles(folder, { exclude = [] } = {}) {
|
|
34
35
|
const skip = new Set(exclude)
|
|
@@ -36,7 +37,7 @@ export async function scanFiles(folder, { exclude = [] } = {}) {
|
|
|
36
37
|
for (const name of await readdir(folder)) {
|
|
37
38
|
if (name.startsWith('.') || skip.has(name)) continue
|
|
38
39
|
const path = join(folder, name)
|
|
39
|
-
const info = await
|
|
40
|
+
const info = await lstat(path)
|
|
40
41
|
if (info.isFile()) {
|
|
41
42
|
entries.push({ name, path, size: info.size, mtimeMs: info.mtimeMs })
|
|
42
43
|
}
|
|
@@ -60,7 +61,10 @@ export async function loadManifest(manifestPath) {
|
|
|
60
61
|
}
|
|
61
62
|
|
|
62
63
|
// Atomic-ish write: write a temp sibling then rename over the target, so a crash
|
|
63
|
-
// during the write can't leave a half-written manifest.
|
|
64
|
+
// during the write can't leave a half-written manifest. Mode 0600: the manifest
|
|
65
|
+
// lists every synced file name with its vault file id, which is nobody else's
|
|
66
|
+
// business on a shared machine. chmod as well as the open mode, because a temp
|
|
67
|
+
// file left by a crashed run keeps whatever mode it had.
|
|
64
68
|
export async function saveManifest(manifestPath, manifest) {
|
|
65
69
|
const body = JSON.stringify(
|
|
66
70
|
{ version: MANIFEST_VERSION, files: manifest.files },
|
|
@@ -68,8 +72,9 @@ export async function saveManifest(manifestPath, manifest) {
|
|
|
68
72
|
2,
|
|
69
73
|
)
|
|
70
74
|
const tmp = `${manifestPath}.tmp`
|
|
71
|
-
const handle = await open(tmp, 'w')
|
|
75
|
+
const handle = await open(tmp, 'w', 0o600)
|
|
72
76
|
try {
|
|
77
|
+
await handle.chmod(0o600)
|
|
73
78
|
await handle.writeFile(body)
|
|
74
79
|
await handle.sync()
|
|
75
80
|
} finally {
|
package/src/upload.mjs
CHANGED
|
@@ -17,7 +17,9 @@
|
|
|
17
17
|
// also carries the header. The header's file_id is the row id the server
|
|
18
18
|
// assigned at create-upload-session, so the stored object is bound to its row.
|
|
19
19
|
|
|
20
|
+
import { constants as fsConstants } from 'node:fs'
|
|
20
21
|
import { open } from 'node:fs/promises'
|
|
22
|
+
import { webcrypto } from 'node:crypto'
|
|
21
23
|
|
|
22
24
|
import { encryptNameV6 } from '@shieldfive/crypto/vault'
|
|
23
25
|
|
|
@@ -25,8 +27,8 @@ import {
|
|
|
25
27
|
base64ToBytes,
|
|
26
28
|
bytesToBase64,
|
|
27
29
|
CIPHER_VERSION_PQ,
|
|
30
|
+
computeV3EmptyUploadProof,
|
|
28
31
|
computeV3UploadProof,
|
|
29
|
-
encryptMetadataV4,
|
|
30
32
|
encryptPartsV3,
|
|
31
33
|
generateRandomKeyB64,
|
|
32
34
|
getCiphertextHashFromParts,
|
|
@@ -41,6 +43,8 @@ import {
|
|
|
41
43
|
const CREATE_SESSION_PATH = '/api/files/create-upload-session'
|
|
42
44
|
const COMPLETE_PATH = '/api/files/complete-upload'
|
|
43
45
|
const PART_URL_PATH = '/api/files/upload-part-url'
|
|
46
|
+
const DELETE_PATH = '/api/files/delete'
|
|
47
|
+
const CAPABILITIES_PATH = '/api/files/upload-capabilities'
|
|
44
48
|
|
|
45
49
|
// Backblaze documents retrying b2_upload_part on 5xx / network failures, and a
|
|
46
50
|
// part upload is idempotent by (part number, SHA-1) — retrying overwrites the
|
|
@@ -56,8 +60,16 @@ const partRetryDelayMs = (attempt) =>
|
|
|
56
60
|
// Read a file in fixed-size chunks without holding it all in memory. Each yield
|
|
57
61
|
// is an independent copy (the read buffer is reused), so the caller may retain
|
|
58
62
|
// or transform a chunk across the next read safely.
|
|
63
|
+
//
|
|
64
|
+
// O_NOFOLLOW: callers list files with lstat and skip symlinks, but a regular
|
|
65
|
+
// file could be swapped for a symlink between that check and this open. With
|
|
66
|
+
// O_NOFOLLOW the open fails (ELOOP) instead of uploading whatever the link
|
|
67
|
+
// points at, possibly outside the folder being pushed. (Undefined on Windows,
|
|
68
|
+
// where it is simply not applied.)
|
|
69
|
+
const OPEN_READ_NOFOLLOW = fsConstants.O_RDONLY | (fsConstants.O_NOFOLLOW ?? 0)
|
|
70
|
+
|
|
59
71
|
export async function* readFileChunks(path, chunkSize) {
|
|
60
|
-
const handle = await open(path,
|
|
72
|
+
const handle = await open(path, OPEN_READ_NOFOLLOW)
|
|
61
73
|
try {
|
|
62
74
|
const buffer = Buffer.allocUnsafe(chunkSize)
|
|
63
75
|
for (;;) {
|
|
@@ -85,18 +97,36 @@ export async function* readFileChunks(path, chunkSize) {
|
|
|
85
97
|
}
|
|
86
98
|
}
|
|
87
99
|
|
|
100
|
+
// A fresh UUIDv7 (RFC 9562): 48-bit ms timestamp, version 7, variant 10.
|
|
101
|
+
// The web app's newRowBoundId (utils/fileRowBinding.ts) builds the same shape;
|
|
102
|
+
// the server's create-upload-session takes it as clientUploadId.
|
|
103
|
+
export function newRowBoundId(now = Date.now()) {
|
|
104
|
+
const bytes = webcrypto.getRandomValues(new Uint8Array(16))
|
|
105
|
+
let ts = BigInt(now)
|
|
106
|
+
for (let i = 5; i >= 0; i -= 1) {
|
|
107
|
+
bytes[i] = Number(ts & 0xffn)
|
|
108
|
+
ts >>= 8n
|
|
109
|
+
}
|
|
110
|
+
bytes[6] = (bytes[6] & 0x0f) | 0x70
|
|
111
|
+
bytes[8] = (bytes[8] & 0x3f) | 0x80
|
|
112
|
+
const hex = Array.from(bytes, (b) => b.toString(16).padStart(2, '0')).join('')
|
|
113
|
+
return `${hex.slice(0, 8)}-${hex.slice(8, 12)}-${hex.slice(12, 16)}-${hex.slice(16, 20)}-${hex.slice(20)}`
|
|
114
|
+
}
|
|
115
|
+
|
|
88
116
|
export async function createUploadSession({
|
|
89
117
|
apiBaseUrl,
|
|
90
118
|
accessToken,
|
|
91
119
|
rootKey,
|
|
92
120
|
name,
|
|
93
121
|
size,
|
|
122
|
+
rowId,
|
|
94
123
|
}) {
|
|
95
124
|
const rootKeyB64 = bytesToBase64(rootKey)
|
|
96
|
-
// The row id
|
|
97
|
-
//
|
|
98
|
-
//
|
|
99
|
-
|
|
125
|
+
// The client picks the row id (UUIDv7, sent as clientUploadId), so the very
|
|
126
|
+
// first stored name is already the v6 envelope bound to that row. No v4
|
|
127
|
+
// (row-unbound) name is ever written, even for an upload that fails before
|
|
128
|
+
// finalize.
|
|
129
|
+
const nameEnvelopeV6 = await encryptNameV6({ name, folderKey: rootKey, rowId })
|
|
100
130
|
// For suite 0x03 this is the classical envelope key; csk_wrapped holds it
|
|
101
131
|
// under the parent key (the root key: sf uploads to the vault root).
|
|
102
132
|
const csk = generateRandomKeyB64()
|
|
@@ -111,7 +141,8 @@ export async function createUploadSession({
|
|
|
111
141
|
// v1 wire format: no cipher_nonce_prefix and no chunkSize (the server
|
|
112
142
|
// rejects both; they live in the header).
|
|
113
143
|
body: JSON.stringify({
|
|
114
|
-
|
|
144
|
+
clientUploadId: rowId,
|
|
145
|
+
name_encrypted: JSON.stringify(nameEnvelopeV6),
|
|
115
146
|
nameHash: hashMetadataV4(name.toLowerCase(), rootKeyB64),
|
|
116
147
|
csk_wrapped: cskEnvelope.wrapped,
|
|
117
148
|
csk_iv: cskEnvelope.iv,
|
|
@@ -129,7 +160,7 @@ export async function createUploadSession({
|
|
|
129
160
|
`create-upload-session failed (HTTP ${res.status}): ${JSON.stringify(body)}`,
|
|
130
161
|
)
|
|
131
162
|
}
|
|
132
|
-
return { session: body, csk }
|
|
163
|
+
return { session: body, csk, nameEncryptedV6: JSON.stringify(nameEnvelopeV6) }
|
|
133
164
|
}
|
|
134
165
|
|
|
135
166
|
// Direct upload: a presigned S3 PUT (web b52b4c8 / PR #726), not b2_upload_file.
|
|
@@ -274,14 +305,22 @@ async function finalizeUpload({
|
|
|
274
305
|
ciphertextHash,
|
|
275
306
|
nameEncryptedV6,
|
|
276
307
|
}) {
|
|
277
|
-
|
|
278
|
-
|
|
279
|
-
|
|
280
|
-
|
|
281
|
-
|
|
282
|
-
|
|
283
|
-
|
|
284
|
-
|
|
308
|
+
let res
|
|
309
|
+
try {
|
|
310
|
+
res = await fetch(new URL(COMPLETE_PATH, apiBaseUrl), {
|
|
311
|
+
method: 'POST',
|
|
312
|
+
headers: {
|
|
313
|
+
Authorization: `Bearer ${accessToken}`,
|
|
314
|
+
'Content-Type': 'application/json',
|
|
315
|
+
},
|
|
316
|
+
body: JSON.stringify({ fileId, b2FileId, partSha1Array, proof, ciphertextHash, nameEncryptedV6 }),
|
|
317
|
+
})
|
|
318
|
+
} catch (err) {
|
|
319
|
+
// The request may have reached the server and finalized the file; only the
|
|
320
|
+
// answer was lost. Cancelling now could bin a file that is in the vault.
|
|
321
|
+
err.finalizeOutcomeUnknown = true
|
|
322
|
+
throw err
|
|
323
|
+
}
|
|
285
324
|
const body = await res.json().catch(() => ({}))
|
|
286
325
|
if (!res.ok) {
|
|
287
326
|
throw new Error(
|
|
@@ -294,7 +333,7 @@ async function finalizeUpload({
|
|
|
294
333
|
// Single-chunk file: one part (header || frame_0), SHA-1 + proof, PUT it at
|
|
295
334
|
// the presigned URL, finalize. No b2FileId is sent -- the server HEADs the
|
|
296
335
|
// stored object and derives the storage id itself.
|
|
297
|
-
async function uploadDirect({ apiBaseUrl, accessToken, session, parts, nameEncryptedV6 }) {
|
|
336
|
+
async function uploadDirect({ apiBaseUrl, accessToken, session, parts, nameEncryptedV6, empty = false }) {
|
|
298
337
|
let ciphertext = null
|
|
299
338
|
for await (const part of parts) {
|
|
300
339
|
if (ciphertext !== null) {
|
|
@@ -303,8 +342,12 @@ async function uploadDirect({ apiBaseUrl, accessToken, session, parts, nameEncry
|
|
|
303
342
|
ciphertext = part
|
|
304
343
|
}
|
|
305
344
|
if (ciphertext === null) throw new Error('Direct upload produced no ciphertext.')
|
|
345
|
+
// For a 0-byte file part 1 is the header alone, and the proof covers it:
|
|
346
|
+
// the server's empty-file contract is HMAC(proofKey, [3][3] || header).
|
|
306
347
|
const sha1 = await sha1Hex(ciphertext)
|
|
307
|
-
const proof =
|
|
348
|
+
const proof = empty
|
|
349
|
+
? computeV3EmptyUploadProof({ proofKeyHex: session.proofKey, header: ciphertext })
|
|
350
|
+
: await computeV3UploadProof({ proofKeyHex: session.proofKey, part0: ciphertext })
|
|
308
351
|
|
|
309
352
|
await putDirectPresigned({ session, ciphertext })
|
|
310
353
|
|
|
@@ -444,21 +487,102 @@ async function uploadMultipart({ apiBaseUrl, accessToken, session, parts, nameEn
|
|
|
444
487
|
})
|
|
445
488
|
}
|
|
446
489
|
|
|
490
|
+
// Best-effort cancel of a session that will never finalize. The pending row
|
|
491
|
+
// otherwise holds quota (and, for multipart, an unfinished Backblaze large
|
|
492
|
+
// file) until the server's stale-upload reaper gets to it. POST
|
|
493
|
+
// /api/files/delete with the row id is what the web app's cancel path uses:
|
|
494
|
+
// for a non-ready row it cancels the large file and drops the metadata. Never
|
|
495
|
+
// throws; a failure here must not hide the error that caused it.
|
|
496
|
+
export async function cancelUploadSession({ apiBaseUrl, accessToken, fileId }) {
|
|
497
|
+
try {
|
|
498
|
+
const res = await fetch(new URL(DELETE_PATH, apiBaseUrl), {
|
|
499
|
+
method: 'POST',
|
|
500
|
+
headers: {
|
|
501
|
+
Authorization: `Bearer ${accessToken}`,
|
|
502
|
+
'Content-Type': 'application/json',
|
|
503
|
+
},
|
|
504
|
+
body: JSON.stringify({ ids: [fileId] }),
|
|
505
|
+
})
|
|
506
|
+
return res.ok
|
|
507
|
+
} catch {
|
|
508
|
+
return false
|
|
509
|
+
}
|
|
510
|
+
}
|
|
511
|
+
|
|
512
|
+
export class EmptyFileUnsupportedError extends Error {
|
|
513
|
+
constructor(name) {
|
|
514
|
+
super(
|
|
515
|
+
`"${name}" is empty (0 bytes) and this ShieldFive server does not accept ` +
|
|
516
|
+
'empty files yet. Nothing was uploaded for it.',
|
|
517
|
+
)
|
|
518
|
+
this.name = 'EmptyFileUnsupportedError'
|
|
519
|
+
this.code = 'empty_file_unsupported'
|
|
520
|
+
}
|
|
521
|
+
}
|
|
522
|
+
|
|
523
|
+
// Zero-byte cipher-v3 uploads are behind a server capability (web
|
|
524
|
+
// docs/desktop-empty-file-contract.md). Ask before creating a session, so an
|
|
525
|
+
// older or flag-off server gets no pending row at all.
|
|
526
|
+
async function serverAcceptsEmptyFiles({ apiBaseUrl, accessToken }) {
|
|
527
|
+
try {
|
|
528
|
+
const res = await fetch(new URL(CAPABILITIES_PATH, apiBaseUrl), {
|
|
529
|
+
headers: { Authorization: `Bearer ${accessToken}` },
|
|
530
|
+
})
|
|
531
|
+
if (!res.ok) return false
|
|
532
|
+
const body = await res.json().catch(() => ({}))
|
|
533
|
+
return body?.emptyCipherV3 === true
|
|
534
|
+
} catch {
|
|
535
|
+
return false
|
|
536
|
+
}
|
|
537
|
+
}
|
|
538
|
+
|
|
447
539
|
// `onPlaintextChunk`, when given, is called with every plaintext chunk in file
|
|
448
540
|
// order, from the same read that feeds encryption. The agent hashes those
|
|
449
541
|
// chunks to record what was actually uploaded; hashing the file separately,
|
|
450
542
|
// before or after, cannot see a change made and undone while it was being read.
|
|
451
|
-
|
|
543
|
+
// `rowId` is for tests only; callers let it default to a fresh UUIDv7.
|
|
544
|
+
export async function uploadFile({ apiBaseUrl, accessToken, rootKey, name, path, size, onPlaintextChunk, rowId = newRowBoundId() }) {
|
|
452
545
|
// Derived before anything is sent, so a vault whose identity cannot be
|
|
453
546
|
// derived fails before a session (and a pending row) exists.
|
|
454
547
|
const recipientPublicKey = await vaultMlKemPublicKey(rootKey)
|
|
455
|
-
|
|
548
|
+
if (size === 0 && !(await serverAcceptsEmptyFiles({ apiBaseUrl, accessToken }))) {
|
|
549
|
+
throw new EmptyFileUnsupportedError(name)
|
|
550
|
+
}
|
|
551
|
+
const { session, csk, nameEncryptedV6 } = await createUploadSession({
|
|
456
552
|
apiBaseUrl,
|
|
457
553
|
accessToken,
|
|
458
554
|
rootKey,
|
|
459
555
|
name,
|
|
460
556
|
size,
|
|
557
|
+
rowId,
|
|
461
558
|
})
|
|
559
|
+
// From here on a pending row exists. Any failure before finalize answers
|
|
560
|
+
// cancels it (see cancelUploadSession).
|
|
561
|
+
const envelopeKey = base64ToBytes(csk)
|
|
562
|
+
try {
|
|
563
|
+
// The name and the file header are bound to rowId. A server that ignored
|
|
564
|
+
// clientUploadId would hand back another id; refuse it (and cancel the
|
|
565
|
+
// row it made) rather than upload under a mismatched binding.
|
|
566
|
+
if (session?.fileId !== rowId) {
|
|
567
|
+
throw new Error(
|
|
568
|
+
`create-upload-session returned file id ${session?.fileId} for requested ${rowId}; ` +
|
|
569
|
+
'this ShieldFive server is too old for this client.',
|
|
570
|
+
)
|
|
571
|
+
}
|
|
572
|
+
await sendUpload({ apiBaseUrl, accessToken, name, path, size, onPlaintextChunk, session, envelopeKey, recipientPublicKey, nameEncryptedV6 })
|
|
573
|
+
} catch (err) {
|
|
574
|
+
if (!err?.finalizeOutcomeUnknown && session?.fileId) {
|
|
575
|
+
await cancelUploadSession({ apiBaseUrl, accessToken, fileId: session.fileId })
|
|
576
|
+
}
|
|
577
|
+
throw err
|
|
578
|
+
} finally {
|
|
579
|
+
envelopeKey.fill(0)
|
|
580
|
+
}
|
|
581
|
+
|
|
582
|
+
return { fileId: session.fileId }
|
|
583
|
+
}
|
|
584
|
+
|
|
585
|
+
async function sendUpload({ apiBaseUrl, accessToken, name, path, size, onPlaintextChunk, session, envelopeKey, recipientPublicKey, nameEncryptedV6 }) {
|
|
462
586
|
if (!session.proofKey) throw new Error('Upload session is missing its proof key.')
|
|
463
587
|
|
|
464
588
|
// The server's v1 chunk size (5 MiB): each encrypted chunk is one storage part.
|
|
@@ -469,7 +593,6 @@ export async function uploadFile({ apiBaseUrl, accessToken, rootKey, name, path,
|
|
|
469
593
|
yield chunk
|
|
470
594
|
}
|
|
471
595
|
}
|
|
472
|
-
const envelopeKey = base64ToBytes(csk)
|
|
473
596
|
const parts = encryptPartsV3({
|
|
474
597
|
plaintextChunks: plaintextChunks(),
|
|
475
598
|
plaintextSize: size,
|
|
@@ -478,34 +601,24 @@ export async function uploadFile({ apiBaseUrl, accessToken, rootKey, name, path,
|
|
|
478
601
|
fileId: uuidToBytes(session.fileId),
|
|
479
602
|
recipientPublicKey,
|
|
480
603
|
})
|
|
481
|
-
// The name
|
|
482
|
-
// finalize,
|
|
483
|
-
|
|
484
|
-
|
|
485
|
-
|
|
486
|
-
|
|
487
|
-
|
|
488
|
-
|
|
489
|
-
|
|
490
|
-
|
|
491
|
-
// the only credential and it already binds the object key. Requiring the
|
|
492
|
-
// token the server stopped issuing is what made every <= 5 MiB upload fail
|
|
493
|
-
// from 2026-09-03. The 'large' branch below still needs both.
|
|
494
|
-
if (!session.uploadUrl) {
|
|
495
|
-
throw new Error('Direct upload session is missing an upload URL.')
|
|
496
|
-
}
|
|
497
|
-
await uploadDirect({ apiBaseUrl, accessToken, session, parts, nameEncryptedV6 })
|
|
498
|
-
} else if (session.uploadKind === 'large') {
|
|
499
|
-
if (!session.uploadUrl || !session.authToken) {
|
|
500
|
-
throw new Error('Multipart upload session is missing credentials.')
|
|
501
|
-
}
|
|
502
|
-
await uploadMultipart({ apiBaseUrl, accessToken, session, parts, nameEncryptedV6 })
|
|
503
|
-
} else {
|
|
504
|
-
throw new Error(`Unknown uploadKind "${session.uploadKind}" for "${name}".`)
|
|
604
|
+
// The same v6 name sent at create (bound to the row id) is sent again at
|
|
605
|
+
// finalize, which still expects it.
|
|
606
|
+
if (session.uploadKind === 'direct') {
|
|
607
|
+
// No authToken and no storagePath check: since web b52b4c8 (PR #726) the
|
|
608
|
+
// direct path is a presigned S3 PUT, so the signature inside uploadUrl is
|
|
609
|
+
// the only credential and it already binds the object key. Requiring the
|
|
610
|
+
// token the server stopped issuing is what made every <= 5 MiB upload fail
|
|
611
|
+
// from 2026-09-03. The 'large' branch below still needs both.
|
|
612
|
+
if (!session.uploadUrl) {
|
|
613
|
+
throw new Error('Direct upload session is missing an upload URL.')
|
|
505
614
|
}
|
|
506
|
-
|
|
507
|
-
|
|
615
|
+
await uploadDirect({ apiBaseUrl, accessToken, session, parts, nameEncryptedV6, empty: size === 0 })
|
|
616
|
+
} else if (session.uploadKind === 'large') {
|
|
617
|
+
if (!session.uploadUrl || !session.authToken) {
|
|
618
|
+
throw new Error('Multipart upload session is missing credentials.')
|
|
619
|
+
}
|
|
620
|
+
await uploadMultipart({ apiBaseUrl, accessToken, session, parts, nameEncryptedV6 })
|
|
621
|
+
} else {
|
|
622
|
+
throw new Error(`Unknown uploadKind "${session.uploadKind}" for "${name}".`)
|
|
508
623
|
}
|
|
509
|
-
|
|
510
|
-
return { fileId: session.fileId }
|
|
511
624
|
}
|
package/src/uploadCrypto.mjs
CHANGED
|
@@ -4,7 +4,7 @@
|
|
|
4
4
|
// default format, written by @shieldfive/crypto (see the end of this file). Same @noble versions as the apps
|
|
5
5
|
// (@noble/ciphers@2, @noble/hashes@2) so the bytes match.
|
|
6
6
|
|
|
7
|
-
import { webcrypto } from 'node:crypto'
|
|
7
|
+
import { createHmac, webcrypto } from 'node:crypto'
|
|
8
8
|
|
|
9
9
|
import { gcm } from '@noble/ciphers/aes.js'
|
|
10
10
|
import { argon2id } from '@noble/hashes/argon2.js'
|
|
@@ -282,7 +282,13 @@ export async function* encryptPartsV3({
|
|
|
282
282
|
}
|
|
283
283
|
const header = take(headerLength)
|
|
284
284
|
const first = await readFrame()
|
|
285
|
-
if (!first)
|
|
285
|
+
if (!first) {
|
|
286
|
+
// A 0-byte file is a header with no chunk frames (total_chunks 0); it is
|
|
287
|
+
// uploaded as a single header-only part. Anything else is a bug.
|
|
288
|
+
if (plaintextSize !== 0) throw new Error('Encrypted stream has no chunks.')
|
|
289
|
+
yield header
|
|
290
|
+
return
|
|
291
|
+
}
|
|
286
292
|
const part0 = new Uint8Array(header.length + first.length)
|
|
287
293
|
part0.set(header, 0)
|
|
288
294
|
part0.set(first, header.length)
|
|
@@ -302,3 +308,21 @@ export async function* encryptPartsV3({
|
|
|
302
308
|
export function computeV3UploadProof({ proofKeyHex, part0 }) {
|
|
303
309
|
return buildUploadProofV3({ proofKeyHex, ciphertext: part0 })
|
|
304
310
|
}
|
|
311
|
+
|
|
312
|
+
/**
|
|
313
|
+
* Upload proof for a 0-byte file, whose only part is the header. The library's
|
|
314
|
+
* buildUploadProofV3 requires a first chunk frame, which an empty file does not
|
|
315
|
+
* have; the server's empty-file contract (web docs/desktop-empty-file-contract.md,
|
|
316
|
+
* emptyFileUploadProof) is base64([3][3] || HMAC-SHA256(proofKey, [3][3] || header)).
|
|
317
|
+
*/
|
|
318
|
+
export function computeV3EmptyUploadProof({ proofKeyHex, header }) {
|
|
319
|
+
if (typeof proofKeyHex !== 'string' || !/^[0-9a-f]{64}$/i.test(proofKeyHex)) {
|
|
320
|
+
throw new Error('Upload session proof key is not 32 bytes of hex.')
|
|
321
|
+
}
|
|
322
|
+
const prefix = Buffer.from([3, 3])
|
|
323
|
+
const mac = createHmac('sha256', Buffer.from(proofKeyHex, 'hex'))
|
|
324
|
+
.update(prefix)
|
|
325
|
+
.update(header)
|
|
326
|
+
.digest()
|
|
327
|
+
return Buffer.concat([prefix, mac]).toString('base64')
|
|
328
|
+
}
|