@shieldfive/cli 0.2.1 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -5,6 +5,71 @@ All notable changes to `@shieldfive/cli` will be documented in this file.
5
5
  The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
6
6
  and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
7
7
 
8
+ ## Unreleased
9
+
10
+ ## 0.3.0 - 2026-09-16
11
+
12
+ ### Added
13
+
14
+ - **`sf login`, `sf status`, `sf logout`.** `sf login` prompts for your password
15
+ and, if two-factor is enabled, a code, then starts an agent that holds the
16
+ signed-in session and the unlocked vault key in memory only. `sf push` and
17
+ `sf sync` use it when it is running and need nothing in the environment. The
18
+ agent locks itself after 8 hours without use (`--idle=<hours>` to change it),
19
+ and `sf logout` revokes the session on the server.
20
+ - **`sf verify <file>...`** reports whether each file is safely stored in your
21
+ vault. The bytes on disk now must match an upload the agent performed, and the
22
+ server must confirm that upload at the moment of asking. Names and sizes play
23
+ no part. Exits 0 only when every file is confirmed, so a script can gate on it.
24
+ - **Unattended sync on two-factor accounts.** One interactive `sf login` is
25
+ enough: the agent refreshes the session without prompting again until it locks
26
+ or you log out.
27
+
28
+ - **`sf logout --everywhere`** signs out every session on the account, including
29
+ the browser and phone.
30
+
31
+ ### Security
32
+
33
+ - The agent's socket accepts six operations: `status`, `upload`, `sync`,
34
+ `verify`, `lock`, `logout`. None lists, downloads or decrypts vault contents,
35
+ or returns key material, and a test pins the list.
36
+ - `sf login` never reads `SF_PASSWORD`. The agent is started with an explicit
37
+ environment allowlist, so a password exported for the older flow does not
38
+ reach it.
39
+ - Uploads are recorded in `~/.shieldfive/ledger/<account>.jsonl` with an HMAC of
40
+ each file's contents under a key derived from the vault key rather than a
41
+ plain hash, so the ledger cannot confirm which documents you hold to anyone
42
+ without that key.
43
+ - Hardened before release after a security review:
44
+ - The ledger records the MAC of the bytes actually encrypted, read in the
45
+ same pass as the upload. Hashing the file before and after could not see a
46
+ change made and undone mid-upload, and `sf verify` would then have called a
47
+ file backed up that the vault did not hold.
48
+ - Ledger records carry a tag under the vault-derived key and a device id;
49
+ forged lines, and lines copied from another machine, are ignored.
50
+ - `SF_EMAIL` and `SF_PASSWORD` take precedence over a running agent, so a
51
+ script cannot be redirected to a different account.
52
+ - A lock file stops two agents from starting at once, and an agent that is
53
+ still revoking its session cannot delete the socket of the one that replaced
54
+ it.
55
+ - `sf status` no longer postpones the idle lock.
56
+ - Stopping wipes keys before touching the network, and its token refresh and
57
+ revoke share a 5 second budget, so an unreachable server cannot keep the
58
+ agent alive. An upload already running keeps its own copy of the key.
59
+ - The socket lives in `~/.shieldfive/run` everywhere, so cron jobs and other
60
+ minimal environments find the agent.
61
+
62
+ ### Notes
63
+
64
+ - `sf login` has not been run against production. The pieces are covered by
65
+ offline tests, including one that runs the agent as its own process against a
66
+ local stub of the auth and verify endpoints. Sign-in, unlock and session
67
+ refresh against the live backend are not.
68
+ - The agent does not run on Windows. `SF_EMAIL` and `SF_PASSWORD` keep working
69
+ there, and everywhere, exactly as before.
70
+ - The ledger starts empty. Existing `.shieldfive-sync.json` manifests are not
71
+ imported: they hold no content hash to check a claim against.
72
+
8
73
  ## 0.2.1 - 2026-09-14
9
74
 
10
75
  ### Fixed
package/README.md CHANGED
@@ -41,6 +41,11 @@ size, uploading nothing.
41
41
  large file.
42
42
  - `sf sync` is **append-only**: it uploads new and changed files; it does not
43
43
  yet mirror local deletions or renames into the vault.
44
+ - **`sf login` and the agent have not been run against production.** They are
45
+ covered by offline tests, including one that runs the agent as its own process
46
+ and reaches it from separate `sf` processes against a local stub of the auth
47
+ and verify endpoints. Sign-in, unlock and session refresh against the live
48
+ backend are not covered.
44
49
 
45
50
  ## Requirements
46
51
 
@@ -66,27 +71,58 @@ trust it):
66
71
  git clone https://github.com/shieldfive/cli.git
67
72
  cd cli
68
73
  npm install # @shieldfive/crypto (+ libsodium), @supabase/supabase-js, @noble/*
69
- npm test # 40/40
74
+ npm test # 107 tests
70
75
  ```
71
76
 
72
77
  ## Commands
73
78
 
74
79
  ```
75
- sf encrypt <folder> encrypt each file locally, upload nothing (no account needed)
76
- sf push <folder> sign in, unlock, encrypt, upload every file once
80
+ sf login [--idle=<hours>] sign in once; an agent holds the session in memory (locks after 8 h idle)
81
+ sf status is an agent running, and for which account
82
+ sf logout [--everywhere] revoke the session and stop the agent; --everywhere signs out every device
83
+ sf push <folder> encrypt and upload every file once
77
84
  sf sync <folder> [--watch] [--interval=N] upload new/changed files; --watch keeps a poll loop (N seconds, default 5)
85
+ sf verify <file>... report whether each file is safely stored in your vault
86
+ sf encrypt <folder> encrypt each file locally, upload nothing (no account needed)
78
87
  ```
79
88
 
80
89
  `sf push` and `sf sync` pick the direct or multipart upload path automatically
81
- from the file size. `sf sync` writes a `.shieldfive-sync.json` manifest in the
82
- target folder that records what has already been uploaded (by size and mtime),
83
- so unchanged files are skipped and an interrupted run does not re-upload
84
- everything.
90
+ from the file size.
91
+
92
+ With an agent running, `sf sync` records uploads in the agent's ledger (see
93
+ [What the agent can and cannot do](#what-the-agent-can-and-cannot-do)) and
94
+ `sf verify` works. Without one, `sf sync` writes a `.shieldfive-sync.json`
95
+ manifest in the target folder that records what has already been uploaded (by
96
+ size and mtime), so unchanged files are skipped and an interrupted run does not
97
+ re-upload everything; `sf verify` needs the agent.
85
98
 
86
99
  ## Running the live commands
87
100
 
88
- `sf push` and `sf sync` need only your ShieldFive account. Your password is
89
- typed, never passed on the command line:
101
+ ### With `sf login`
102
+
103
+ ```sh
104
+ sf login
105
+ sf push ./my-folder
106
+ sf sync ./my-folder --watch
107
+ sf verify ./my-folder/tax-return-2025.pdf
108
+ sf logout
109
+ ```
110
+
111
+ `sf login` prompts for your email, your password and, if it differs, your vault
112
+ password. It never reads them from the environment and there is no flag for
113
+ them. It then starts an agent in the background and exits; every later command
114
+ talks to that agent and needs nothing exported.
115
+
116
+ If `SF_EMAIL` and `SF_PASSWORD` are also set, `sf push` and `sf sync` sign in
117
+ with those instead and tell you an agent is running, so a script written for one
118
+ account is never sent to another. `sf push` exits 2 if a file changed while it
119
+ was being uploaded: the vault then holds the version that was read, not the file
120
+ as it is now.
121
+
122
+ ### Without an agent
123
+
124
+ `sf push` and `sf sync` still work from environment variables, exactly as
125
+ before. Your password is typed, never passed on the command line:
90
126
 
91
127
  ```sh
92
128
  export SF_EMAIL=you@example.com
@@ -116,6 +152,68 @@ Enter the current code from your authenticator. In a non-interactive context
116
152
  (scripts, CI) set `SF_TOTP_CODE` to the current code instead of being prompted.
117
153
  Only TOTP is supported; SMS and other factors are not.
118
154
 
155
+ With `sf login` you are asked once. The agent refreshes the stepped-up session
156
+ on its own, so `sf sync --watch` keeps working on a two-factor account until the
157
+ agent locks or you log out.
158
+
159
+ ## What the agent can and cannot do
160
+
161
+ The agent keeps two things in memory: your signed-in session and your unlocked
162
+ vault key. Neither is written to disk. It listens on a Unix socket in
163
+ `~/.shieldfive/run`, a directory only your user can enter, and refuses to start
164
+ if that directory is a symlink, owned by someone else or open to other users.
165
+ Only one agent runs at a time.
166
+
167
+ It accepts six requests: `status`, `upload`, `sync`, `verify`, `lock` and
168
+ `logout`. It cannot list what is in your vault, download a file, decrypt
169
+ anything, or hand out the key, because there is no request for any of those. A
170
+ process that reaches the socket can put files into your vault and ask whether
171
+ files are backed up. It cannot read your vault through the agent.
172
+
173
+ Uploads are recorded in `~/.shieldfive/ledger/<account>.jsonl`: the path, size,
174
+ time, vault file id, and an HMAC of the bytes that were actually encrypted, under
175
+ a key derived from your vault key. A plain hash would let anyone holding the
176
+ ledger confirm whether you have a particular known document; the HMAC does not,
177
+ without your vault key. Each record also carries a tag under that key, so a line
178
+ added to the ledger by anything other than the agent is ignored, and records
179
+ made on another machine do not count on this one.
180
+
181
+ `sf verify` reports a file as **backed up** only when the bytes on disk now
182
+ match an upload in the ledger and the server confirms, at that moment, that the
183
+ upload is stored, complete, intact and not in your Bin. It never decides by name
184
+ or size. The other answers:
185
+
186
+ | Answer | Meaning |
187
+ |---|---|
188
+ | `NOT confirmed` | these bytes were uploaded, but the server cannot confirm they are still safely stored |
189
+ | `changed since upload` | this path was uploaded, but the file has changed since |
190
+ | `no upload record` | this agent never uploaded these bytes; they may still be in your vault from another device |
191
+ | `unreadable` | the local file could not be read |
192
+
193
+ `no upload record` does not mean the file is missing from your vault. Do not
194
+ delete anything on the strength of that answer.
195
+
196
+ The agent locks itself after 8 hours without an upload, sync or verify
197
+ (`sf login --idle=<hours>` to change it; `sf status` does not count). Locking and
198
+ `sf logout` both overwrite the key in memory first and then sign the session out
199
+ on the server. Sessions on ShieldFive do not expire on their own, so a session
200
+ that is forgotten rather than signed out would stay valid. If the server cannot
201
+ be reached when you log out, `sf logout` says the session is still valid and
202
+ exits 1; run `sf login` and then `sf logout --everywhere` once you are online.
203
+
204
+ Limits:
205
+
206
+ - Windows is not supported. Use `SF_EMAIL` and `SF_PASSWORD` there.
207
+ - Node cannot see which process connected to the socket, so directory
208
+ permissions are the only gate. This is also how `ssh-agent` works on macOS.
209
+ - Overwriting the key in memory is best-effort. A garbage-collected runtime can
210
+ copy key material during crypto operations.
211
+ - The ledger is per device. A file uploaded from another machine shows as
212
+ `no upload record` here.
213
+
214
+ The full design, including the threat model, is in
215
+ [`docs/agent-design.md`](docs/agent-design.md).
216
+
119
217
  ## How the upload works (for auditors)
120
218
 
121
219
  The upload uses **chunked AES-GCM** for file content and issues a
@@ -167,6 +265,12 @@ file — a machine-checked demonstration that only ciphertext is uploaded.
167
265
  finalize; the streaming chunk reader
168
266
  - `src/sync.mjs` — `sf sync`: manifest, change detection, reconcile pass, watch
169
267
  loop (reuses `upload.mjs`)
268
+ - `src/prompt.mjs` — terminal prompts; the password prompt echoes nothing
269
+ - `src/agent/` — `sf login`'s agent: `server.mjs` (the process and its six
270
+ requests), `client.mjs`, `protocol.mjs` (the request allowlist), `ledger.mjs`
271
+ (HMAC upload ledger and `verify` classification), `paths.mjs` (socket
272
+ directory checks), `main.mjs` (entry point and session handoff)
273
+ - `docs/agent-design.md` — design record and threat model for the agent
170
274
  - `src/sfCrypto.mjs` — the post-quantum hybrid encrypt/decrypt used by the
171
275
  `sf encrypt` demo
172
276
  - `test/*.test.mjs` — round-trip guarantees plus the mock end-to-end upload and
package/SECURITY.md CHANGED
@@ -63,9 +63,13 @@ not pursue civil claims or refer law enforcement against researchers who:
63
63
 
64
64
  ## Bug bounty
65
65
 
66
- ShieldFive operates a paid bug bounty program. For current scope, reward tiers,
67
- rules of engagement, and submission instructions, see
68
- https://shieldfive.com/security/bug-bounty.
66
+ **There is no bug bounty.** ShieldFive ran one until 2026-07; it closed, and
67
+ `/security/bug-bounty` now redirects to the security page. Reports are answered
68
+ on the timelines above, and credited if you want credit. They are not paid.
69
+
70
+ This section previously said the opposite and pointed at that redirected URL.
71
+ It is corrected here rather than removed, so anyone who read the old text finds
72
+ out what changed.
69
73
 
70
74
  ## Scope
71
75
 
@@ -79,6 +83,6 @@ Out of scope for *this* repository (report elsewhere or not at all):
79
83
  [`@shieldfive/crypto`](https://github.com/shieldfive/crypto).
80
84
  - Vulnerabilities in dependencies (`@noble/*`, `@supabase/supabase-js`) — report
81
85
  those upstream.
82
- - Server-side issues in the ShieldFive backend — report via the bug bounty
83
- program above.
86
+ - Server-side issues in the ShieldFive backend — same address,
87
+ `security@shieldfive.com`, different codebase.
84
88
  - Attacks that require an attacker to already control the user's device.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@shieldfive/cli",
3
- "version": "0.2.1",
3
+ "version": "0.3.0",
4
4
  "type": "module",
5
5
  "description": "Open-source command-line client for ShieldFive: encrypt a folder on your machine and sync it to your EU-hosted, post-quantum vault. Read the code; it encrypts before upload.",
6
6
  "license": "Apache-2.0",
@@ -0,0 +1,102 @@
1
+ // ShieldFive CLI — talk to a running agent.
2
+
3
+ import { connect } from 'node:net'
4
+
5
+ import { socketPath } from './paths.mjs'
6
+ import { encode, lineSplitter } from './protocol.mjs'
7
+
8
+ export class AgentUnavailableError extends Error {
9
+ constructor(message) {
10
+ super(message)
11
+ this.name = 'AgentUnavailableError'
12
+ this.code = 'agent_unavailable'
13
+ }
14
+ }
15
+
16
+ export class AgentRequestError extends Error {
17
+ constructor(code, message) {
18
+ super(message)
19
+ this.name = 'AgentRequestError'
20
+ this.code = code
21
+ }
22
+ }
23
+
24
+ /**
25
+ * Send one request and resolve with the result.
26
+ *
27
+ * `timeoutMs: 0` disables the timeout, which uploads and syncs need: a large
28
+ * file can take longer than any fixed number.
29
+ */
30
+ export function request(op, args = {}, { path, timeoutMs = 10_000 } = {}) {
31
+ return new Promise((resolve, reject) => {
32
+ // Resolved here rather than as a default parameter, so a platform the agent
33
+ // does not support rejects this promise instead of throwing out of it.
34
+ let socketFile
35
+ try {
36
+ socketFile = path ?? socketPath()
37
+ } catch (err) {
38
+ reject(err)
39
+ return
40
+ }
41
+
42
+ let settled = false
43
+ const socket = connect({ path: socketFile })
44
+
45
+ const settle = (fn, value) => {
46
+ if (settled) return
47
+ settled = true
48
+ clearTimeout(timer)
49
+ socket.destroy()
50
+ fn(value)
51
+ }
52
+
53
+ const timer = timeoutMs
54
+ ? setTimeout(
55
+ () => settle(reject, new AgentRequestError('timeout', `Agent did not answer within ${timeoutMs} ms.`)),
56
+ timeoutMs,
57
+ )
58
+ : null
59
+
60
+ socket.on('error', (err) => {
61
+ if (err.code === 'ENOENT' || err.code === 'ECONNREFUSED') {
62
+ settle(reject, new AgentUnavailableError('The ShieldFive agent is not running. Run sf login.'))
63
+ } else {
64
+ settle(reject, err)
65
+ }
66
+ })
67
+
68
+ socket.setEncoding('utf8')
69
+ socket.on(
70
+ 'data',
71
+ lineSplitter((line) => {
72
+ let message
73
+ try {
74
+ message = JSON.parse(line)
75
+ } catch {
76
+ return settle(reject, new AgentRequestError('bad_response', 'Agent sent something that is not JSON.'))
77
+ }
78
+ if (message.ok) settle(resolve, message.result)
79
+ else settle(reject, new AgentRequestError(message.error?.code ?? 'error', message.error?.message ?? 'Agent refused.'))
80
+ }),
81
+ )
82
+ socket.on('close', () =>
83
+ settle(reject, new AgentRequestError('closed', 'Agent closed the connection without answering.')),
84
+ )
85
+
86
+ socket.on('connect', () => socket.write(encode({ id: 1, op, args })))
87
+ })
88
+ }
89
+
90
+ /**
91
+ * The agent's status, or null when no agent is listening or the agent cannot
92
+ * run on this platform. Callers fall back to environment credentials on null,
93
+ * so an unsupported platform must answer null, not throw.
94
+ */
95
+ export async function agentStatus(options = {}) {
96
+ try {
97
+ return await request('status', {}, { timeoutMs: 2_000, ...options })
98
+ } catch (err) {
99
+ if (err instanceof AgentUnavailableError || err.code === 'unsupported_platform') return null
100
+ throw err
101
+ }
102
+ }
@@ -0,0 +1,254 @@
1
+ // ShieldFive CLI — the upload ledger.
2
+ //
3
+ // One JSON line per upload the agent performed:
4
+ //
5
+ // {"v":1,"path":"/abs/path","size":48211,"mtimeMs":…,"mac":"<hex>",
6
+ // "fileId":"…","uploadedAt":"…","device":"<uuid>","tag":"<hex>"}
7
+ //
8
+ // `mac` is HMAC-SHA256 over exactly the plaintext bytes that were encrypted and
9
+ // uploaded, keyed by a key derived from the vault root key. A plain SHA-256
10
+ // would let anyone who reads this file confirm whether you hold a specific
11
+ // known document; keyed, the entries are unlinkable without the root key.
12
+ //
13
+ // `tag` is HMAC-SHA256 over the record's other fields under the same key. The
14
+ // ledger is an ordinary file the user's processes can write, so without it a
15
+ // forged line naming a missing file's MAC and some real fileId would make that
16
+ // file report as backed up. Records whose tag does not verify are dropped.
17
+ //
18
+ // Nothing here decides that a file is backed up by its name, path or size. A
19
+ // match on the MAC says those exact bytes were uploaded; whether they are still
20
+ // safely stored is a question for the server, asked at the moment it matters
21
+ // (see classify()).
22
+
23
+ import { createHmac, hkdfSync, timingSafeEqual } from 'node:crypto'
24
+ import { createReadStream } from 'node:fs'
25
+ import { mkdir, open, readFile, stat } from 'node:fs/promises'
26
+ import { dirname, isAbsolute, join } from 'node:path'
27
+
28
+ export const LEDGER_VERSION = 1
29
+ const LEDGER_INFO = 'shieldfive/cli/ledger/v1'
30
+ const TAGGED_FIELDS = ['v', 'path', 'size', 'mtimeMs', 'mac', 'fileId', 'uploadedAt', 'device']
31
+ export const VERIFY_BATCH = 100
32
+
33
+ /** Derive the ledger MAC key from the 32-byte vault root key. */
34
+ export function deriveLedgerKey(rootKey) {
35
+ if (!(rootKey instanceof Uint8Array) || rootKey.length !== 32) {
36
+ throw new Error('deriveLedgerKey needs the 32-byte root key')
37
+ }
38
+ return Buffer.from(hkdfSync('sha256', rootKey, Buffer.alloc(0), LEDGER_INFO, 32))
39
+ }
40
+
41
+ /** HMAC-SHA256 of a file's bytes, streamed. Returns lowercase hex. */
42
+ export function macFile(path, ledgerKey) {
43
+ return new Promise((resolve, reject) => {
44
+ const hmac = createHmac('sha256', ledgerKey)
45
+ const stream = createReadStream(path)
46
+ stream.on('error', reject)
47
+ stream.on('data', (chunk) => hmac.update(chunk))
48
+ stream.on('end', () => resolve(hmac.digest('hex')))
49
+ })
50
+ }
51
+
52
+ function tagFor(ledgerKey, record) {
53
+ const fields = TAGGED_FIELDS.map((f) => record[f] ?? null)
54
+ return createHmac('sha256', ledgerKey).update(JSON.stringify(fields)).digest('hex')
55
+ }
56
+
57
+ export function ledgerPath(dir, accountId) {
58
+ // Supabase user ids are UUIDs. Validating the shape keeps an account id from
59
+ // ever being read as a path.
60
+ if (typeof accountId !== 'string' || !/^[0-9a-f-]{36}$/i.test(accountId)) {
61
+ throw new Error(`Refusing ledger for malformed account id ${JSON.stringify(accountId)}`)
62
+ }
63
+ return join(dir, `${accountId.toLowerCase()}.jsonl`)
64
+ }
65
+
66
+ /** Tag, append and fsync one record. Creates the file 0600 inside a 0700 dir. */
67
+ export async function appendEntry(file, entry, { ledgerKey } = {}) {
68
+ if (!ledgerKey) throw new Error('appendEntry needs the ledger key to tag the record')
69
+
70
+ // Build and tag the record before the first await. The agent overwrites the
71
+ // key on logout, and a tag computed after that would be a tag under zeros.
72
+ const record = { v: LEDGER_VERSION, ...entry }
73
+ record.tag = tagFor(ledgerKey, record)
74
+ const line = JSON.stringify(record) + '\n'
75
+
76
+ await mkdir(dirname(file), { recursive: true, mode: 0o700 })
77
+ const handle = await open(file, 'a+', 0o600)
78
+ try {
79
+ // If a previous write was torn and left no trailing newline, start on a new
80
+ // line. Otherwise this record would be glued to the broken one and both
81
+ // would be discarded as malformed.
82
+ const { size } = await handle.stat()
83
+ let prefix = ''
84
+ if (size > 0) {
85
+ const { buffer } = await handle.read(Buffer.alloc(1), 0, 1, size - 1)
86
+ if (buffer[0] !== 0x0a) prefix = '\n'
87
+ }
88
+ await handle.write(prefix + line)
89
+ await handle.sync()
90
+ } finally {
91
+ await handle.close()
92
+ }
93
+ return record
94
+ }
95
+
96
+ /**
97
+ * Every well-formed, correctly tagged record, oldest first.
98
+ *
99
+ * A malformed line or a record whose tag does not verify is counted and
100
+ * skipped rather than failing the read, so one torn write or one forged line
101
+ * does not hide every other upload.
102
+ */
103
+ export async function readEntries(file, { ledgerKey } = {}) {
104
+ if (!ledgerKey) throw new Error('readEntries needs the ledger key to check record tags')
105
+
106
+ let raw
107
+ try {
108
+ raw = await readFile(file, 'utf8')
109
+ } catch (err) {
110
+ if (err.code === 'ENOENT') return { entries: [], malformed: 0, unauthenticated: 0 }
111
+ throw err
112
+ }
113
+
114
+ const entries = []
115
+ let malformed = 0
116
+ let unauthenticated = 0
117
+ for (const line of raw.split('\n')) {
118
+ if (!line.trim()) continue
119
+ let rec
120
+ try {
121
+ rec = JSON.parse(line)
122
+ } catch {
123
+ malformed++
124
+ continue
125
+ }
126
+ const wellFormed =
127
+ rec?.v === LEDGER_VERSION &&
128
+ typeof rec.path === 'string' &&
129
+ typeof rec.mac === 'string' &&
130
+ /^[0-9a-f]{64}$/.test(rec.mac) &&
131
+ typeof rec.fileId === 'string' &&
132
+ rec.fileId &&
133
+ typeof rec.tag === 'string' &&
134
+ /^[0-9a-f]{64}$/.test(rec.tag)
135
+ if (!wellFormed) {
136
+ malformed++
137
+ continue
138
+ }
139
+ const expected = Buffer.from(tagFor(ledgerKey, rec), 'hex')
140
+ if (!timingSafeEqual(expected, Buffer.from(rec.tag, 'hex'))) {
141
+ unauthenticated++
142
+ continue
143
+ }
144
+ entries.push(rec)
145
+ }
146
+ return { entries, malformed, unauthenticated }
147
+ }
148
+
149
+ /**
150
+ * Where each local path stands.
151
+ *
152
+ * verified the bytes on disk now are bytes that were uploaded,
153
+ * and the server confirms that upload is stored, ready,
154
+ * outside the Bin and intact
155
+ * missing the bytes were uploaded, but the server cannot confirm
156
+ * it is still safely stored
157
+ * changed_since_upload this path was uploaded, but the file has changed since
158
+ * not_in_ledger no record of these bytes being uploaded from here
159
+ * unreadable the local file could not be read
160
+ *
161
+ * `not_in_ledger` is not `missing`. It means this agent never uploaded these
162
+ * bytes; the file may well be in the vault from another device.
163
+ *
164
+ * When `device` is given, only records made on that device count.
165
+ *
166
+ * `verifyFileIds(ids)` must resolve to the ids the server reports verified. It
167
+ * is called in batches of at most VERIFY_BATCH, and an id in an answer only
168
+ * counts if that batch asked about it.
169
+ */
170
+ export async function classify(paths, { entries, ledgerKey, verifyFileIds, device }) {
171
+ const usable = device ? entries.filter((e) => e.device === device) : entries
172
+
173
+ const byMac = new Map()
174
+ const byPath = new Map()
175
+ for (const e of usable) {
176
+ const macList = byMac.get(e.mac) ?? []
177
+ macList.push(e)
178
+ byMac.set(e.mac, macList)
179
+ byPath.set(e.path, e) // later records win: entries are oldest first
180
+ }
181
+
182
+ const pending = []
183
+ const results = []
184
+
185
+ for (const path of paths) {
186
+ if (typeof path !== 'string' || !isAbsolute(path)) {
187
+ results.push({ path, state: 'unreadable', reason: 'not an absolute path' })
188
+ continue
189
+ }
190
+
191
+ let st
192
+ try {
193
+ st = await stat(path)
194
+ } catch (err) {
195
+ results.push({ path, state: 'unreadable', reason: err.code ?? 'stat failed' })
196
+ continue
197
+ }
198
+ if (!st.isFile()) {
199
+ results.push({ path, state: 'unreadable', reason: 'not a regular file' })
200
+ continue
201
+ }
202
+
203
+ let mac
204
+ try {
205
+ mac = await macFile(path, ledgerKey)
206
+ } catch (err) {
207
+ results.push({ path, state: 'unreadable', reason: err.code ?? 'read failed' })
208
+ continue
209
+ }
210
+
211
+ const sameBytes = byMac.get(mac)
212
+ if (sameBytes?.length) {
213
+ // Prefer the record for this exact path; otherwise any upload of the same
214
+ // bytes counts, because identity here is the content, not the location.
215
+ const own = sameBytes.filter((e) => e.path === path)
216
+ const candidates = (own.length ? own : sameBytes).map((e) => e.fileId)
217
+ const result = { path, state: null, size: st.size, fileIds: [...new Set(candidates)] }
218
+ if (!own.length) result.matchedPath = sameBytes[sameBytes.length - 1].path
219
+ results.push(result)
220
+ pending.push(result)
221
+ continue
222
+ }
223
+
224
+ if (byPath.has(path)) {
225
+ results.push({ path, state: 'changed_since_upload', size: st.size })
226
+ continue
227
+ }
228
+
229
+ results.push({ path, state: 'not_in_ledger', size: st.size })
230
+ }
231
+
232
+ const ids = [...new Set(pending.flatMap((r) => r.fileIds))]
233
+ const verified = new Set()
234
+ for (let i = 0; i < ids.length; i += VERIFY_BATCH) {
235
+ const batch = ids.slice(i, i + VERIFY_BATCH)
236
+ const asked = new Set(batch)
237
+ const answer = await verifyFileIds(batch)
238
+ for (const id of answer ?? []) {
239
+ // An id the server volunteered without being asked in this batch is not
240
+ // an answer to this question. On a check that gates deletion, that must
241
+ // fail closed.
242
+ if (asked.has(id)) verified.add(id)
243
+ }
244
+ }
245
+
246
+ for (const r of pending) {
247
+ const ok = r.fileIds.find((id) => verified.has(id))
248
+ r.state = ok ? 'verified' : 'missing'
249
+ if (ok) r.fileId = ok
250
+ delete r.fileIds
251
+ }
252
+
253
+ return results
254
+ }