keychat-save 1.4.2 → 1.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -83,6 +83,40 @@ A bundle carries only committed history. It misses uncommitted work, gitignored
83
83
  and `node_modules` — which means a restore depends on the npm registry still serving every package
84
84
  at the same version. Packages get unpublished. RepoChain carries the lot.
85
85
 
86
+ ## KeyCloud — files on chain
87
+
88
+ `save` handles text and **RepoChain** handles repositories. **KeyCloud** puts a single file on
89
+ chain in one transaction, sealed to your own key. Nobody else can read it, including us.
90
+
91
+ ```bash
92
+ # Upload — one file, one transaction
93
+ keychat-save keycloud put ~/Documents/passport.pdf
94
+
95
+ # Fetch it back anywhere your identity key is. sha256 is verified before
96
+ # anything is written to disk.
97
+ keychat-save keycloud get <txid> ~/Downloads
98
+
99
+ # What you have uploaded from this machine
100
+ keychat-save keycloud list
101
+ ```
102
+
103
+ Bytes are encrypted directly — never base64, which would add 33% to the size and therefore 33% to
104
+ the cost. The file's sha256 travels with it and is checked on the way back, so a truncated fetch
105
+ fails loudly instead of writing a corrupt file.
106
+
107
+ ### What it costs
108
+
109
+ Storage is priced like permanence, at the 100 sats/KB mining floor:
110
+
111
+ | File | Cost |
112
+ |------|------|
113
+ | 1 MB | ~200 Compute |
114
+ | 10 MB | ~2,000 Compute |
115
+ | 100 MB | ~20,000 Compute (0.1 BSV) |
116
+
117
+ `put` refuses anything over 95 MB rather than spend on a transaction no miner will accept — miner
118
+ policy caps a single transaction at 100 MB.
119
+
86
120
  ## How it works
87
121
 
88
122
  Content is sealed with your identity key (ECIES) and broadcast as a transaction on BSV via
@@ -18,3 +18,30 @@ if (!readFileSync(new URL('./index.mjs', import.meta.url), 'utf8').includes('100
18
18
  console.error('Without it an underpaid save is accepted, never mined, and lost hours later.')
19
19
  process.exit(1)
20
20
  }
21
+
22
+ // EVERY RELATIVE IMPORT MUST RESOLVE INSIDE THE PACKAGE.
23
+ //
24
+ // 1.4.3 SHIPPED WITH REPOCHAIN COMPLETELY DEAD. repo.mjs imported
25
+ // './spent-ledger-fs.js' and '../src/wallet/spent-ledger.js' — neither was in
26
+ // the package — so every `keychat-save repo ...` exited with ERR_MODULE_NOT_FOUND
27
+ // before doing anything. The two guards above both passed: repo.mjs WAS
28
+ // generated and index.mjs DID have the floor guard. Nothing checked that the
29
+ // package could actually load.
30
+ for (const f of ['repo.mjs', 'index.mjs', 'spent-ledger.js', 'spent-ledger-fs.js']) {
31
+ let body
32
+ try { body = readFileSync(new URL(`./${f}`, import.meta.url), 'utf8') } catch { continue }
33
+ for (const m of body.matchAll(/from\s+'(\.[^']+)'/g)) {
34
+ try {
35
+ readFileSync(new URL(m[1], new URL(`./${f}`, import.meta.url)), 'utf8')
36
+ } catch {
37
+ console.error(`${f} imports '${m[1]}', which is not in the package.`)
38
+ console.error('That is what shipped broken in 1.4.3 — the CLI cannot even load.')
39
+ process.exit(1)
40
+ }
41
+ }
42
+ }
43
+
44
+ // And the guard the other two could not give: actually LOAD the modules. A
45
+ // syntax error in the generated file (a bad cut point produces one) passes
46
+ // every textual check above and still bricks the CLI.
47
+ await import('./repo.mjs')
package/index.mjs CHANGED
@@ -75,8 +75,8 @@ async function getUnspent (address) {
75
75
  // The bridge answers in its own shape when our node can serve the tx:
76
76
  // { outputs: [ { satoshis, script, index } ] }
77
77
  // When it CANNOT — the node is pruned with no -txindex, so getrawtransaction fails for
78
- // any confirmed tx in a discarded block — the web proxy falls back to WhatsOnChain and
79
- // returns WoC's body untouched:
78
+ // any confirmed tx in a discarded block — the bridge falls through to GorillaPool and
79
+ // the body comes back in the other shape:
80
80
  // { vout: [ { value, n, scriptPubKey: { hex } } ] }
81
81
  //
82
82
  // Callers looked only at `tx.outputs`, so on the WoC shape they iterated an empty array
@@ -259,6 +259,33 @@ async function cmdSave (label, body) {
259
259
  }
260
260
  console.log(` Fee: ${sum - changeSats} sats / ${rawHex.length / 2} bytes = ${rate.toFixed(2)} sats/KB`)
261
261
 
262
+ // FRESH READ, IN THE SAME COMMAND, IMMEDIATELY BEFORE THE BROADCAST.
263
+ //
264
+ // Everything above planned against the getUnspent call at the top of this
265
+ // function, and that snapshot is now as old as building and signing took.
266
+ // Another writer on the same key — the KeyChat web app, a second terminal,
267
+ // a repo snapshot — can have taken an input in that window. Broadcasting
268
+ // anyway produces a tx that is accepted by some peers, never mined, and gone
269
+ // hours later: exactly the silent-loss failure the fee floor above guards
270
+ // against, arriving by a different route.
271
+ //
272
+ // THE CHAIN IS THE LEDGER. The bridge verifies every outpoint against the
273
+ // node's chainstate with include_mempool=true — "is this usable for a new tx
274
+ // right now" — so anything another writer has already BROADCAST is absent
275
+ // from this response. No reservation table, no shared state: just look again
276
+ // before committing.
277
+ const fresh = await getUnspent(address)
278
+ const live = new Set(fresh.map(u => `${u.txid}:${u.vout}`))
279
+ const gone = picked.filter(u => !live.has(`${u.txid}:${u.vout}`))
280
+ if (gone.length) {
281
+ throw new Error(
282
+ `${gone.length} of ${picked.length} input(s) were spent while this save was ` +
283
+ `being built (first: ${gone[0].txid.slice(0, 16)}…:${gone[0].vout}). ` +
284
+ `NOT broadcast — it would have been a dead double-spend. Re-run; the ` +
285
+ `picker will take fresh coins.`
286
+ )
287
+ }
288
+
262
289
  const txid = await broadcast(rawHex)
263
290
 
264
291
  // Prepend new entry to local saves index
@@ -449,6 +476,12 @@ if (cmd === 'init') {
449
476
  // RepoChain — whole git repositories on chain, not just session text.
450
477
  const { runRepo } = await import('./repo.mjs')
451
478
  await runRepo(args)
479
+ } else if (cmd === 'keycloud') {
480
+ // KeyCloud — a single file on chain, sealed to your own key. Shares repo.mjs
481
+ // because it shares publish(): the funding-tx pattern, the 100 sats/KB floor,
482
+ // the self-marker, and the pre-broadcast chain re-read.
483
+ const { runKeycloud } = await import('./repo.mjs')
484
+ await runKeycloud(args)
452
485
  } else {
453
486
  console.log(`keychat-save — Save to BSV blockchain via KeyChat (keychat.pro)
454
487
 
@@ -469,6 +502,11 @@ REPOCHAIN — whole git repositories:
469
502
  repo estimate <path> Size and cost, broadcasts nothing
470
503
  repo list Snapshots taken from this machine
471
504
 
505
+ KEYCLOUD — files on chain, sealed to your own key:
506
+ keycloud put <file> Upload one file in a single transaction
507
+ keycloud get <txid> <dest> Fetch it back (sha256 verified before writing)
508
+ keycloud list Files uploaded from this machine
509
+
472
510
  Each snapshot is ONE transaction. A delta continues the chain automatically and
473
511
  carries the ordered txids of every ancestor, so a single txid restores everything.
474
512
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "keychat-save",
3
- "version": "1.4.2",
3
+ "version": "1.5.0",
4
4
  "description": "Save sessions, files and whole git repositories to the BSV blockchain via KeyChat",
5
5
  "type": "module",
6
6
  "bin": {
@@ -26,7 +26,7 @@
26
26
  "license": "MIT",
27
27
  "repository": {
28
28
  "type": "git",
29
- "url": "https://github.com/avaziri93/keychat-save"
29
+ "url": "git+https://github.com/avaziri93/keychat-save.git"
30
30
  },
31
31
  "homepage": "https://keychat.pro",
32
32
  "scripts": {
package/repo.mjs CHANGED
@@ -48,6 +48,14 @@ import { createHash } from 'node:crypto'
48
48
  import fs from 'node:fs'
49
49
  import os from 'node:os'
50
50
  import path from 'node:path'
51
+ import { installFileLedger } from './spent-ledger-fs.js'
52
+ import { markSpentOutpoints, withoutSpent, pruneConfirmed, rebuildFromNode } from './spent-ledger.js'
53
+
54
+ // Share one spent-ledger with keychat-save and every other Node writer on this
55
+ // machine. A repo snapshot and a session save draw from the SAME wallet; a
56
+ // snapshot can spend many seconds building a multi-MB tx, and without this a
57
+ // save started in that window picks the same inputs and dies as a double-spend.
58
+ installFileLedger()
51
59
 
52
60
  const CONFIG_DIR = path.join(os.homedir(), '.keychat')
53
61
  const WIF_PATH = path.join(CONFIG_DIR, 'identity.wif')
@@ -75,14 +83,71 @@ function key () {
75
83
  }
76
84
 
77
85
  // ── bridge ───────────────────────────────────────────────────────
78
- async function getUnspent (address) {
86
+ async function getUnspent (address, { skipLedger = false } = {}) {
79
87
  const res = await fetch(`${BRIDGE}/api/address/${address}/unspent`)
80
88
  if (!res.ok) throw new Error(`unspent ${res.status}`)
81
- return (await res.json()).map(u => ({
89
+ const mapped = (await res.json()).map(u => ({
82
90
  txid: u.tx_hash || u.txid,
83
91
  vout: u.tx_pos ?? u.vout,
84
92
  satoshis: u.value ?? u.satoshis
85
93
  }))
94
+ if (skipLedger) return mapped
95
+ // The node says what exists; the ledger only subtracts coins another writer
96
+ // has already claimed. Prune each load so it stays the size of what is in
97
+ // flight rather than growing forever.
98
+ pruneConfirmed(mapped)
99
+ return withoutSpent(mapped)
100
+ }
101
+
102
+ /**
103
+ * A snapshot that never reached the network still holds its coins. Re-derive
104
+ * from a fresh node scan rather than un-marking by hand — the node is
105
+ * authoritative at this moment, and a hand-rolled release can be wrong in the
106
+ * direction that double-spends. Failure path only.
107
+ */
108
+ async function releaseLedgerAfterFailure (address) {
109
+ try {
110
+ const fresh = await getUnspent(address, { skipLedger: true })
111
+ const r = rebuildFromNode(fresh)
112
+ console.warn(` ledger rebuilt from node: ${r.released} released, ${r.dropped} dropped`)
113
+ } catch (err) {
114
+ console.warn(` could not rebuild ledger: ${err.message}`)
115
+ }
116
+ }
117
+
118
+ /**
119
+ * Thrown when an input we signed was taken before we broadcast. Distinct type
120
+ * so the caller can rebuild on fresh coins instead of shipping a dead tx.
121
+ */
122
+ class StaleInputsError extends Error {}
123
+
124
+ /**
125
+ * LAST READ BEFORE BROADCAST — are the coins we just signed still ours?
126
+ *
127
+ * publish() reads /unspent once at the top and then does real work: planning,
128
+ * building and signing, tens of seconds for a multi-MB payload. Any other
129
+ * writer on this key can take an input in that window — most obviously the
130
+ * browser, whose spent-ledger lives in localStorage and cannot see this
131
+ * file-backed one. Nothing looked again, so a snapshot could be signed against
132
+ * coins that were already gone and broadcast dead. That is how 811d165c lost
133
+ * two of its 76 inputs.
134
+ *
135
+ * THE CHAIN IS THE LEDGER HERE, not local state. The bridge's /unspent verifies
136
+ * every outpoint against the node's chainstate with include_mempool=true —
137
+ * "is this usable for a new tx right now" — so anything another writer has
138
+ * BROADCAST is already absent from this response. That is the same read Compute
139
+ * History reconstructs the whole wallet from. No reservation table needed.
140
+ *
141
+ * skipLedger: true is REQUIRED. markSpentOutpoints(picked) has already claimed
142
+ * these coins in our own ledger, so the filtered view would report every one of
143
+ * them missing and this check would fire on every single save.
144
+ *
145
+ * Returns the outpoints that are gone; empty means clear to broadcast.
146
+ */
147
+ async function staleInputs (address, picked) {
148
+ const fresh = await getUnspent(address, { skipLedger: true })
149
+ const live = new Set(fresh.map(u => `${u.txid}:${u.vout}`))
150
+ return picked.filter(u => !live.has(`${u.txid}:${u.vout}`))
86
151
  }
87
152
 
88
153
  async function broadcast (rawHex) {
@@ -103,24 +168,36 @@ async function broadcast (rawHex) {
103
168
  // 68 MB of hex in the live web server to satisfy a restore is not worth it. A query
104
169
  // string makes its isTxBody regex miss, so the request passes straight through.
105
170
  //
106
- // But that same regex miss also skips its WhatsOnChain fallback, which is the only
107
- // second source we have once a tx is mined. So fall back to the plain path on
108
- // failure retrievability beats keeping the proxy cache tidy.
171
+ // The plain path is still tried on failure retrievability beats keeping the
172
+ // proxy cache tidy. (It used to also carry the proxy's WhatsOnChain fallback;
173
+ // that was removed 2026-08-28, so the third source below is now the only
174
+ // second opinion once a tx is mined.)
109
175
  async function getRawHex (txid) {
110
176
  // Three sources, in order of sovereignty:
111
177
  // 1. bridge ?raw=1 — our node, bypassing the web proxy's forever-cache.
112
- // 2. bridge plain — same node, but with the proxy's WoC fallback behind it.
113
- // 3. WoC direct — last resort, and it earns its place: our node is PRUNED
114
- // with no -txindex, so getrawtransaction CANNOT return a confirmed tx at
115
- // all. Every mined snapshot is therefore unfetchable from us, and with
116
- // only the first two sources a rate-limited proxy made the whole delta
117
- // chain unreplayable — which breaks restore, not just new deltas.
118
- // 429 gets a backoff rather than being treated as absence; hammering a closed
119
- // limiter is what turned a transient failure into a hard one.
178
+ // 2. bridge plain — same node; the bridge itself falls through to GP.
179
+ // 3. GorillaPool raw — last resort, and it earns its place: our node is
180
+ // PRUNED with no -txindex, so getrawtransaction CANNOT return a
181
+ // confirmed tx at all. Every mined snapshot is unfetchable from us
182
+ // directly, and without a third source a rate-limited proxy made the
183
+ // whole delta chain unreplayable — which breaks restore, not just new
184
+ // deltas.
185
+ //
186
+ // WAS WhatsOnChain until 2026-08-28. WoC TRUNCATES LARGE SCRIPTS, and this
187
+ // function fetches repo snapshots — the multi-MB transactions most likely to
188
+ // be cut short. A truncated body does not error; it decrypts to garbage or
189
+ // fails a hash check, so the failure surfaces as a corrupt restore rather
190
+ // than a fetch error. It also throttled at ~3 req/s, and the responses that
191
+ // lost were the biggest ones. GP serves the same transactions whole and
192
+ // faster — measured: a 4.8 MB snapshot in 3.4s.
193
+ //
194
+ // GP returns raw BYTES, so convert rather than reading text as hex.
195
+ // 429 still gets a backoff rather than being treated as absence; hammering a
196
+ // closed limiter is what turned a transient failure into a hard one.
120
197
  const urls = [
121
198
  `${BRIDGE}/api/tx/${txid}/hex?raw=1`,
122
199
  `${BRIDGE}/api/tx/${txid}/hex`,
123
- `https://api.whatsonchain.com/v1/bsv/main/tx/${txid}/hex`
200
+ `https://ordinals.gorillapool.io/api/tx/${txid}/raw`
124
201
  ]
125
202
  let wait = 500
126
203
  for (let attempt = 0; attempt < 3; attempt++) {
@@ -129,6 +206,15 @@ async function getRawHex (txid) {
129
206
  try { res = await fetch(url) } catch { continue }
130
207
  if (res.status === 429) { await new Promise(r => setTimeout(r, wait)); continue }
131
208
  if (!res.ok) continue
209
+ // GP's /raw returns BINARY tx bytes, not hex text. Reading it with
210
+ // .text() yields mojibake that fails the hex test below, so the source
211
+ // would silently never work — a fallback that is present but dead is
212
+ // worse than no fallback, because nothing tells you.
213
+ if (url.endsWith('/raw')) {
214
+ const h = Buffer.from(await res.arrayBuffer()).toString('hex')
215
+ if (h && /^[0-9a-f]+$/i.test(h)) return h
216
+ continue
217
+ }
132
218
  const ct = res.headers.get('content-type') || ''
133
219
  const data = ct.includes('json') ? await res.json() : await res.text()
134
220
  const h = (typeof data === 'string' ? data.replace(/"/g, '').trim() : data.hex)
@@ -137,7 +223,7 @@ async function getRawHex (txid) {
137
223
  wait = Math.min(wait * 4, 8000)
138
224
  await new Promise(r => setTimeout(r, wait))
139
225
  }
140
- throw new Error(`cannot fetch raw tx ${txid} from node, bridge or WoC`)
226
+ throw new Error(`cannot fetch raw tx ${txid} from node, bridge or GorillaPool`)
141
227
  }
142
228
 
143
229
  // ── OP_RETURN ────────────────────────────────────────────────────
@@ -347,6 +433,29 @@ function planFunding (utxos, target) {
347
433
  // delta small enough to skip it has nothing to lose.
348
434
  const DIRECT_MAX_INPUTS = 60
349
435
 
436
+ /**
437
+ * publish(), retried on fresh coins if an input was taken mid-build.
438
+ *
439
+ * Re-entering publish() re-reads /unspent from the top, so the rebuild plans
440
+ * against the current chain rather than the snapshot that just went stale. The
441
+ * payload script is unchanged — only the funding is re-planned — so a retry
442
+ * costs a rebuild and a re-sign, never a different snapshot.
443
+ *
444
+ * Bounded at 3 attempts. If coins are being taken out from under us that
445
+ * persistently, something else is wrong and looping would just burn fees.
446
+ */
447
+ async function publishWithRetry (scriptHex, k, address, lockingScript, opts = {}) {
448
+ for (let attempt = 1; ; attempt++) {
449
+ try {
450
+ return await publish(scriptHex, k, address, lockingScript, opts)
451
+ } catch (err) {
452
+ if (!(err instanceof StaleInputsError) || attempt >= 3) throw err
453
+ console.log(`\n ${err.message}`)
454
+ console.log(` rebuilding on fresh coins (attempt ${attempt + 1}/3) ...\n`)
455
+ }
456
+ }
457
+ }
458
+
350
459
  /** Broadcast one OP_RETURN payload, funding it whichever way is cheaper. */
351
460
  async function publish (scriptHex, k, address, lockingScript, { dry = false } = {}) {
352
461
  const opReturn = Script.fromHex(scriptHex)
@@ -403,6 +512,17 @@ async function publish (scriptHex, k, address, lockingScript, { dry = false } =
403
512
  console.log(`payload tx ${(bytes / 1024).toFixed(1)} KB, ${n} inputs, ` +
404
513
  `fee ${fee} sats = ${fee / SATS_PER_COMPUTE} Compute (single tx, ${rate.toFixed(2)} sats/KB)`)
405
514
  if (dry) return { txid: tx.id('hex'), funding: null, cost: fee / SATS_PER_COMPUTE, dry: true }
515
+ // Same fresh read as the two-tx path below. This path claims nothing in the
516
+ // ledger, so the inputs are simply the ones we built with.
517
+ process.stdout.write('re-checking inputs ... ')
518
+ const goneDirect = await staleInputs(address, utxos.slice(0, n))
519
+ if (goneDirect.length) {
520
+ console.log(`${goneDirect.length} of ${n} TAKEN`)
521
+ throw new StaleInputsError(
522
+ `${goneDirect.length} input(s) were spent while this tx was being built ` +
523
+ `(first: ${goneDirect[0].txid.slice(0, 16)}…:${goneDirect[0].vout})`)
524
+ }
525
+ console.log(`${n}/${n} still live`)
406
526
  process.stdout.write('broadcasting ... ')
407
527
  const txid = await broadcast(tx.toHex())
408
528
  console.log(txid)
@@ -431,6 +551,11 @@ async function publish (scriptHex, k, address, lockingScript, { dry = false } =
431
551
  const fundingValue = payloadFee + SELF_MARKER_SATS
432
552
  const { picked, sum: fsum, need } = planFunding(utxos, fundingValue)
433
553
  const fundFee = need - fundingValue
554
+ // Claim the inputs before signing. A multi-MB payload can spend tens of
555
+ // seconds in sign(), and any save started in that window would otherwise
556
+ // pick the very same coins. Released below on every path that does not
557
+ // broadcast the funding tx.
558
+ markSpentOutpoints(picked)
434
559
  console.log(`payload tx ${(payloadBytes / 1048576).toFixed(2)} MB, ` +
435
560
  `fee ${payloadFee} sats = ${payloadFee / SATS_PER_COMPUTE} Compute`)
436
561
  console.log(`funding tx ${picked.length} inputs, fee ${fundFee} sats`)
@@ -458,18 +583,49 @@ async function publish (scriptHex, k, address, lockingScript, { dry = false } =
458
583
 
459
584
  // Both must clear the floor BEFORE either goes out. Broadcasting the funding tx and
460
585
  // then discovering the payload is underpaid strands the fee in a spent output.
461
- assertFeeRate(fundTx, fundFee, 'funding tx')
462
- // Miner receives the funding value MINUS the marker — check the floor against
463
- // what the miner actually gets, not against what the tx consumes.
464
- assertFeeRate(payTx, fundingValue - SELF_MARKER_SATS, 'payload tx')
586
+ try {
587
+ assertFeeRate(fundTx, fundFee, 'funding tx')
588
+ // Miner receives the funding value MINUS the marker check the floor against
589
+ // what the miner actually gets, not against what the tx consumes.
590
+ assertFeeRate(payTx, fundingValue - SELF_MARKER_SATS, 'payload tx')
591
+ } catch (err) {
592
+ await releaseLedgerAfterFailure(address)
593
+ throw err
594
+ }
465
595
 
466
596
  if (dry) {
597
+ // Nothing goes to the network on a dry run, so the coins are still ours.
598
+ await releaseLedgerAfterFailure(address)
467
599
  return { txid: payTx.id('hex'), funding: fundTx.id('hex'), cost: (fundingValue + fundFee) / SATS_PER_COMPUTE, dry: true }
468
600
  }
601
+ // FRESH READ, IN THE SAME COMMAND, IMMEDIATELY BEFORE THE BROADCAST.
602
+ // Everything above this line is planning against a snapshot that is now as
603
+ // old as the signing took. Ask the chain once more before committing.
604
+ process.stdout.write('re-checking inputs ... ')
605
+ const gone = await staleInputs(address, picked)
606
+ if (gone.length) {
607
+ console.log(`${gone.length} of ${picked.length} TAKEN`)
608
+ await releaseLedgerAfterFailure(address)
609
+ throw new StaleInputsError(
610
+ `${gone.length} input(s) were spent while this tx was being built ` +
611
+ `(first: ${gone[0].txid.slice(0, 16)}…:${gone[0].vout})`)
612
+ }
613
+ console.log(`${picked.length}/${picked.length} still live`)
614
+
469
615
  process.stdout.write('broadcasting funding tx ... ')
470
- const ftx = await broadcast(fundTx.toHex())
616
+ let ftx
617
+ try {
618
+ ftx = await broadcast(fundTx.toHex())
619
+ } catch (err) {
620
+ // Funding never left, so `picked` is still ours.
621
+ await releaseLedgerAfterFailure(address)
622
+ throw err
623
+ }
471
624
  console.log(ftx)
472
625
  process.stdout.write('broadcasting payload tx ... ')
626
+ // Deliberately NO release if the payload fails: the funding tx is already on
627
+ // the network, so `picked` is genuinely spent. Releasing here would hand the
628
+ // picker coins that are gone — the one direction that double-spends.
473
629
  const ptx = await broadcast(payTx.toHex())
474
630
  console.log(ptx)
475
631
  return { txid: ptx, funding: ftx, cost: (fundingValue + fundFee) / SATS_PER_COMPUTE }
@@ -584,7 +740,7 @@ async function backup (repoPath, { dry = false } = {}) {
584
740
  sha256: sha, manifestRoot, tarSize: tar.length, at: Math.floor(Date.now() / 1000)
585
741
  }
586
742
  const scriptHex = sealPayload(meta, tar, k, pubHex)
587
- const res = await publish(scriptHex, k, address, lockingScript, { dry })
743
+ const res = await publishWithRetry(scriptHex, k, address, lockingScript, { dry })
588
744
  if (dry) {
589
745
  console.log(`\n--dry: nothing broadcast\n payload ${res.txid}` +
590
746
  (res.funding ? `\n funding ${res.funding}` : ''))
@@ -677,7 +833,7 @@ async function delta (repoPath, baseTxid, { dry = false } = {}) {
677
833
  manifestRoot: curM.root, tarSize: tar.length, at: Math.floor(Date.now() / 1000)
678
834
  }
679
835
  const scriptHex = sealPayload(meta, tar, k, pubHex)
680
- const res = await publish(scriptHex, k, address, lockingScript, { dry })
836
+ const res = await publishWithRetry(scriptHex, k, address, lockingScript, { dry })
681
837
  if (dry) {
682
838
  console.log(`\n--dry: nothing broadcast\n payload ${res.txid}`)
683
839
  return
@@ -723,6 +879,145 @@ async function restore (txid, dest) {
723
879
  console.log(`restored to ${path.join(dest, name)}`)
724
880
  }
725
881
 
882
+ // ── KeyCloud ─────────────────────────────────────────────────────
883
+ // ONE FILE, ONE TRANSACTION, SEALED TO YOUR OWN KEY.
884
+ //
885
+ // KeyCloud is personal storage, not messaging: the file is encrypted to the
886
+ // wallet's OWN public key, so only this identity can ever read it back. It
887
+ // reuses publish() wholesale — the funding-tx pattern, the 100 sats/KB floor,
888
+ // the self-marker, and the pre-broadcast chain re-read — because a second copy
889
+ // of that machinery is precisely how getTx and getRawTxHex drifted apart.
890
+ //
891
+ // KCLD, NOT KSAV OR KREP. A distinct prefix keeps files out of `load` (which
892
+ // would otherwise decrypt every upload into saves.json and then carry it in
893
+ // every future bootstrap table) and out of `repo list`. Same reason KREP was
894
+ // split from KSAV in the first place.
895
+ //
896
+ // RAW BYTES. The file is framed and encrypted directly, never base64 — that
897
+ // 33% tax is real money here: at 100 sats/KB a 10 MB file is ~2,000 Compute,
898
+ // so base64 alone would cost ~660 Compute extra per upload.
899
+ //
900
+ // SIZE. Single tx up to the miner policy ceiling — GorillaPool and TAAL both
901
+ // publish maxtxsizepolicy=100,000,000, and 33.87 MB repo snapshots already go
902
+ // on chain whole this way. Our own bitcoind runs maxscriptsizepolicy=6000000
903
+ // and will not relay past 6 MB itself; ARC carries those and the node picks
904
+ // them up when mined. Above the ceiling the file must be chunked — see the
905
+ // KACH chunk+manifest format in src/messages/attachment.js, whose receive side
906
+ // already exists. put refuses rather than guessing, so nothing is spent on a
907
+ // transaction no miner will accept.
908
+ const CLOUD_PREFIX = '4b434c44' // "KCLD"
909
+ const CLOUD_INDEX = 'keycloud.json'
910
+ // Policy ceiling for a single transaction. The payload is the whole tx, so this
911
+ // is deliberately conservative against the published 100 MB: envelope, ECIES
912
+ // overhead and the funding input all sit inside the same limit.
913
+ const CLOUD_MAX_SINGLE_TX = 95 * 1024 * 1024
914
+
915
+ function recordCloudFile (rec) {
916
+ const idx = path.join(CONFIG_DIR, CLOUD_INDEX)
917
+ let all = []
918
+ try { all = JSON.parse(fs.readFileSync(idx, 'utf8')) } catch {}
919
+ all.push(rec)
920
+ fs.writeFileSync(idx, JSON.stringify(all, null, 2), 'utf8')
921
+ }
922
+
923
+ /** Upload one file to chain, sealed to this wallet's own key. */
924
+ async function cloudPut (filePath, { dry = false } = {}) {
925
+ if (!filePath) throw new Error('usage: keycloud put <file>')
926
+ const abs = path.resolve(filePath)
927
+ if (!fs.existsSync(abs) || !fs.statSync(abs).isFile()) {
928
+ throw new Error(`not a file: ${abs}`)
929
+ }
930
+ const k = key()
931
+ const address = k.toPublicKey().toAddress()
932
+ const pubHex = k.toPublicKey().toString()
933
+ const lockingScript = new P2PKH().lock(address)
934
+
935
+ const bytes = fs.readFileSync(abs)
936
+ if (bytes.length > CLOUD_MAX_SINGLE_TX) {
937
+ throw new Error(
938
+ `${(bytes.length / 1048576).toFixed(1)} MB exceeds the ${CLOUD_MAX_SINGLE_TX / 1048576} MB ` +
939
+ `single-transaction ceiling. Chunked upload (KACH manifest) is not wired into this ` +
940
+ `command yet — nothing was spent.`)
941
+ }
942
+ const sha = createHash('sha256').update(bytes).digest('hex')
943
+ const meta = {
944
+ v: 1,
945
+ kind: 'file',
946
+ name: path.basename(abs),
947
+ size: bytes.length,
948
+ sha256: sha,
949
+ at: Math.floor(Date.now() / 1000)
950
+ }
951
+ console.log(`${meta.name} ${(bytes.length / 1048576).toFixed(2)} MB`)
952
+ console.log(`sha256 ${sha}`)
953
+
954
+ process.stdout.write('encrypting ... ')
955
+ const t = Date.now()
956
+ const cipher = EncryptedMessage.encrypt(
957
+ Array.from(framePayload(meta, bytes)), k, PublicKey.fromString(pubHex)
958
+ )
959
+ console.log(`${((Date.now() - t) / 1000).toFixed(1)}s`)
960
+ const timestamp = Math.floor(Date.now() / 1000).toString(16).padStart(8, '0')
961
+ const scriptHex = buildOpReturn([CLOUD_PREFIX, VERSION, pubHex, hex(cipher), timestamp])
962
+
963
+ const res = await publishWithRetry(scriptHex, k, address, lockingScript, { dry })
964
+ if (dry) {
965
+ console.log(`\n--dry: nothing broadcast\n payload ${res.txid}` +
966
+ (res.funding ? `\n funding ${res.funding}` : ''))
967
+ return
968
+ }
969
+ recordCloudFile({ txid: res.txid, funding: res.funding, ...meta, cost: res.cost })
970
+ console.log(`\nKEYCLOUD ${meta.name} -> ${res.txid}`)
971
+ console.log(`get with: keychat-save keycloud get ${res.txid} <dest>`)
972
+ }
973
+
974
+ /**
975
+ * Fetch one KCLD file back and write it to dest.
976
+ *
977
+ * Verifies the sha256 recorded at upload before writing anything. That check is
978
+ * not ceremony: getRawHex falls through to a third party for anything our
979
+ * pruned node cannot serve, and a TRUNCATED body does not error — it decrypts
980
+ * to garbage. The hash is what turns that into a failure instead of a corrupt
981
+ * file on disk.
982
+ */
983
+ async function cloudGet (txid, dest) {
984
+ if (!/^[0-9a-f]{64}$/.test(txid || '')) throw new Error('usage: keycloud get <txid> <dest>')
985
+ const k = key()
986
+ const pubHex = k.toPublicKey().toString()
987
+ const rawHex = await getRawHex(txid)
988
+ const tx = Transaction.fromHex(rawHex)
989
+ let parts = null
990
+ for (const o of tx.outputs) {
991
+ const s = o.lockingScript.toHex()
992
+ if (s.startsWith('006a') || s.startsWith('6a')) { parts = extractParts(s); break }
993
+ }
994
+ if (!parts || parts[0] !== CLOUD_PREFIX) throw new Error(`${txid} is not a KCLD keycloud transaction`)
995
+ if (parts[2] !== pubHex) throw new Error(`${txid} is not encrypted to this identity`)
996
+ const plain = Buffer.from(EncryptedMessage.decrypt(Array.from(Buffer.from(parts[3], 'hex')), k))
997
+ const { meta, tar: bytes } = unframePayload(plain)
998
+
999
+ const got = createHash('sha256').update(bytes).digest('hex')
1000
+ if (meta.sha256 && got !== meta.sha256) {
1001
+ throw new Error(`sha256 mismatch — expected ${meta.sha256}, got ${got}. NOT written.`)
1002
+ }
1003
+ // A bare directory means "put it back under its own name".
1004
+ let out = path.resolve(dest || '.')
1005
+ if (fs.existsSync(out) && fs.statSync(out).isDirectory()) out = path.join(out, meta.name)
1006
+ fs.writeFileSync(out, bytes)
1007
+ console.log(`${meta.name} ${(bytes.length / 1048576).toFixed(2)} MB sha256 ok`)
1008
+ console.log(`wrote ${out}`)
1009
+ }
1010
+
1011
+ function cloudList () {
1012
+ let all = []
1013
+ try { all = JSON.parse(fs.readFileSync(path.join(CONFIG_DIR, CLOUD_INDEX), 'utf8')) } catch {}
1014
+ if (!all.length) return console.log('no keycloud files recorded locally')
1015
+ for (const r of all.sort((a, b) => b.at - a.at)) {
1016
+ console.log(`${new Date(r.at * 1000).toISOString().slice(0, 10)} ${String(r.name).padEnd(28)} ` +
1017
+ `${(r.size / 1048576).toFixed(2)} MB ${r.cost}C ${r.txid}`)
1018
+ }
1019
+ }
1020
+
726
1021
  function list () {
727
1022
  let all = []
728
1023
  try { all = JSON.parse(fs.readFileSync(path.join(CONFIG_DIR, 'repos.json'), 'utf8')) } catch {}
@@ -755,3 +1050,24 @@ export async function runRepo (argv) {
755
1050
  console.log(' keychat-save repo list')
756
1051
  process.exitCode = 1
757
1052
  }
1053
+
1054
+ /**
1055
+ * CLI entry for the keycloud commands, called from index.mjs.
1056
+ *
1057
+ * Shares this module with runRepo on purpose: KeyCloud uses the same publish()
1058
+ * — funding-tx pattern, 100 sats/KB floor, self-marker, pre-broadcast chain
1059
+ * re-read. A separate module would mean a second copy of that machinery.
1060
+ */
1061
+ export async function runKeycloud (argv) {
1062
+ const [sub, ...rest] = argv
1063
+ const dry = rest.includes('--dry')
1064
+ const args = rest.filter(x => !x.startsWith('--'))
1065
+ if (sub === 'put') return cloudPut(args[0], { dry })
1066
+ if (sub === 'get') return cloudGet(args[0], args[1])
1067
+ if (sub === 'list') return cloudList()
1068
+ console.log('usage:')
1069
+ console.log(' keychat-save keycloud put <file> [--dry] one tx, sealed to your own key')
1070
+ console.log(' keychat-save keycloud get <txid> <dest>')
1071
+ console.log(' keychat-save keycloud list')
1072
+ process.exitCode = 1
1073
+ }
@@ -0,0 +1,49 @@
1
+ // File-backed store for the shared spent-ledger, for every Node entry point:
2
+ // the bot, keychat-save, keychat-repo.
3
+ //
4
+ // The browser keeps its ledger in localStorage; Node has no localStorage (and
5
+ // the bot's shim discards writes), so anything running outside a browser must
6
+ // install this before it spends.
7
+ //
8
+ // WHAT SHARING THIS FILE BUYS
9
+ // Every Node process on one machine reads and writes the same ledger, so a
10
+ // repo snapshot cannot pick the coin a session save just claimed, and neither
11
+ // can pick one the other is mid-broadcast on. That is the "failed saves while
12
+ // the app is also sending" case: two writers, one wallet, one pool.
13
+ //
14
+ // WHERE IT STOPS
15
+ // A browser cannot write to ~/.keychat, so the app's ledger and this one are
16
+ // separate stores. Contention BETWEEN the app and a CLI is still arbitrated by
17
+ // the node alone — which is sufficient there only because a person switching
18
+ // between the app and a terminal takes seconds, against a measured node lag of
19
+ // 10-65ms. It is the sub-100ms machine-speed case that needs a ledger, and
20
+ // that only ever happens within one process or between two Node ones.
21
+ import fs from 'node:fs'
22
+ import os from 'node:os'
23
+ import path from 'node:path'
24
+ import { setLedgerBackend } from './spent-ledger.js'
25
+
26
+ const LEDGER = process.env.KC_SPENT_LEDGER ||
27
+ path.join(os.homedir(), '.keychat', 'spent-ledger.json')
28
+
29
+ export const fileBackend = {
30
+ read () {
31
+ try { return JSON.parse(fs.readFileSync(LEDGER, 'utf8')) } catch { return {} }
32
+ },
33
+ // Temp file + rename: a crash mid-write must not leave truncated JSON, since
34
+ // a failed parse falls back to {} and would silently un-spend everything.
35
+ write (m) {
36
+ fs.mkdirSync(path.dirname(LEDGER), { recursive: true })
37
+ const tmp = `${LEDGER}.${process.pid}.tmp`
38
+ fs.writeFileSync(tmp, JSON.stringify(m))
39
+ fs.renameSync(tmp, LEDGER)
40
+ }
41
+ }
42
+
43
+ /** Point the shared ledger at the filesystem. Call once, before any spend. */
44
+ export function installFileLedger () {
45
+ setLedgerBackend(fileBackend)
46
+ return LEDGER
47
+ }
48
+
49
+ export const LEDGER_PATH = LEDGER
@@ -0,0 +1,195 @@
1
+ // Durable spent-ledger for every KeyChat picker.
2
+ //
3
+ // THE PROBLEM IT SOLVES
4
+ // Two sends fired back-to-back spend the same coin. Measured on 2026-08-28
5
+ // against the bot, which sends twice per reply (the message, then the self-CC
6
+ // copy 0.1ms later): our own node takes 10-65ms to report the first tx's
7
+ // inputs as spent. The second send asks "what can I spend?" inside that
8
+ // window, gets a stale-but-honest answer, and because every Compute piece is
9
+ // exactly 500 sats and the picker is largest-first, it deterministically picks
10
+ // THE SAME COIN. Two txs spending one input; the network keeps one, the other
11
+ // dies silently.
12
+ //
13
+ // The client hits the same thing on any paired flow — delete then
14
+ // re-introduce, accept then auto-reply — which is what caused the 2026-06-02
15
+ // "intros not broadcasting" incident.
16
+ //
17
+ // No amount of node-side correctness fixes this. The node is not wrong, it is
18
+ // BEHIND, and we ask faster than any index can update.
19
+ //
20
+ // THE FIX
21
+ // Record the spend locally AT PICK TIME, before the tx is even signed. The
22
+ // next pick reads this ledger, not the network, so the race has nowhere to
23
+ // happen. This replaces the old in-memory recentlySpent map, whose TTL was an
24
+ // arbitrary constant (10s in the client, 60s on the bot) tuned against a lag
25
+ // nobody had measured — and which was lost entirely on reload or restart.
26
+ //
27
+ // DIRECTIONALITY — this file may only ever REMOVE coins from the pool.
28
+ // Nothing enters the pool from here; the node decides what exists and what is
29
+ // confirmed. A stale flag costs one coin, which is recoverable. Resurrecting a
30
+ // coin the chain has already seen spent is a double-spend, which is not.
31
+ //
32
+ // The ONE way a flag is cleared is rebuildFromNode(), and only after a
33
+ // broadcast FAILED — see that function for why it must never run on success.
34
+ //
35
+ // STORAGE IS INJECTABLE AND MUST BE SYNCHRONOUS. Async storage (IndexedDB)
36
+ // would re-open the very window this closes: the write has to land before the
37
+ // next pick reads. Browser uses localStorage; the bot installs a file-backed
38
+ // store at boot, because its localStorage shim discards writes.
39
+
40
+ const KEY = 'keychat-spent-ledger'
41
+
42
+ const localStorageBackend = {
43
+ read () {
44
+ try { return JSON.parse(localStorage.getItem(KEY) || '{}') } catch { return {} }
45
+ },
46
+ write (m) {
47
+ try { localStorage.setItem(KEY, JSON.stringify(m)) } catch { /* quota — see prune note */ }
48
+ }
49
+ }
50
+
51
+ // A Web Worker has NO localStorage. The send worker imports sendMessage, so it
52
+ // reaches this module — and a backend that silently discards writes there would
53
+ // be WORSE than the in-memory map this replaced, which at least worked inside
54
+ // the worker's own session. So when there is no localStorage, fall back to
55
+ // memory rather than to nothing.
56
+ //
57
+ // The worker still echoes its spentOutpoints to the main thread, which mirrors
58
+ // them into the durable ledger (see workers/send-worker.js) — that is what
59
+ // makes a worker send visible to the next main-thread send. This fallback only
60
+ // has to cover picks made back-to-back INSIDE one worker.
61
+ function makeMemoryBackend () {
62
+ let store = {}
63
+ return { read: () => store, write: (m) => { store = m } }
64
+ }
65
+
66
+ function defaultBackend () {
67
+ try {
68
+ if (typeof localStorage === 'undefined') return makeMemoryBackend()
69
+ // Availability is not the same as usability — Safari private mode throws
70
+ // on setItem rather than on access. Probe once, at load.
71
+ const probe = '__kc_ledger_probe__'
72
+ localStorage.setItem(probe, '1')
73
+ localStorage.removeItem(probe)
74
+ return localStorageBackend
75
+ } catch {
76
+ return makeMemoryBackend()
77
+ }
78
+ }
79
+
80
+ let backend = defaultBackend()
81
+
82
+ /**
83
+ * Install a different synchronous store. The bot calls this with an fs-backed
84
+ * implementation before importing anything that spends. Must expose
85
+ * read(): object and write(object): void, both synchronous.
86
+ */
87
+ export function setLedgerBackend (b) { backend = b }
88
+
89
+ const op = (txid, vout) => `${txid}:${vout}`
90
+
91
+ /**
92
+ * Mark outpoints spent so no picker hands them out again. Call at PICK time,
93
+ * not after broadcast — the whole point is to beat the node, and a
94
+ * post-broadcast write is already too late for the very next send.
95
+ */
96
+ export function markSpentOutpoints (utxos) {
97
+ const m = backend.read()
98
+ const ts = Math.floor(Date.now() / 1000)
99
+ for (const u of utxos) m[op(u.txid, u.vout)] = ts
100
+ backend.write(m)
101
+ }
102
+
103
+ /** True if the ledger has recorded this outpoint as spent. */
104
+ export function isSpentOutpoint (txid, vout) {
105
+ return backend.read()[op(txid, vout)] !== undefined
106
+ }
107
+
108
+ /**
109
+ * Filter a node-supplied UTXO list down to what the ledger still considers
110
+ * unspent. One read for the whole list rather than per-entry.
111
+ */
112
+ export function withoutSpent (utxos) {
113
+ const m = backend.read()
114
+ // FAST PATH — nothing claimed means nothing to filter, so do no work at all.
115
+ // This runs on every getUnspent, which the balance poll calls every 5s, on
116
+ // the main thread, against a wallet that can hold tens of thousands of
117
+ // UTXOs. Building a key string per UTXO there stalls the tab; the ledger
118
+ // meanwhile is empty or holds a handful of in-flight outpoints. Cost must
119
+ // scale with the LEDGER, not the wallet.
120
+ if (isEmpty(m)) return utxos
121
+ return utxos.filter(u => m[op(u.txid, u.vout)] === undefined)
122
+ }
123
+
124
+ // Emptiness without allocating an Object.keys array.
125
+ function isEmpty (m) {
126
+ for (const _k in m) return false
127
+ return true
128
+ }
129
+
130
+ /**
131
+ * Drop entries the node no longer lists — they are confirmed spent, gone from
132
+ * /unspent forever, and their flags are dead weight. Runs on every load so the
133
+ * ledger stays the size of what is actually in flight (a handful of outpoints)
134
+ * rather than growing for the life of the wallet.
135
+ *
136
+ * SAFE ON THE SUCCESS PATH, unlike rebuildFromNode: this only ever forgets
137
+ * coins the node has already dropped, so it cannot hand a spent coin back to
138
+ * the picker. It never touches an entry the node still lists — that is exactly
139
+ * the in-flight case the ledger exists to remember.
140
+ *
141
+ * Writes only when something actually changed, so the common no-op load costs
142
+ * one read.
143
+ */
144
+ export function pruneConfirmed (nodeUtxos) {
145
+ const m = backend.read()
146
+ // Same reasoning as withoutSpent: with an empty ledger there is nothing to
147
+ // prune, and building a Set over every UTXO the wallet owns to discover that
148
+ // is the expensive way to do nothing.
149
+ if (isEmpty(m)) return 0
150
+ const live = new Set(nodeUtxos.map(u => op(u.txid, u.vout)))
151
+ let removed = 0
152
+ for (const k of Object.keys(m)) {
153
+ if (!live.has(k)) { delete m[k]; removed++ }
154
+ }
155
+ if (removed) backend.write(m)
156
+ return removed
157
+ }
158
+
159
+ /**
160
+ * Re-derive the ledger from a FRESH scan of the node's UTXO set.
161
+ *
162
+ * Anything the node still lists is un-marked; anything it no longer lists is
163
+ * dropped, since a confirmed spend is gone from /unspent forever and its flag
164
+ * is dead weight. So one call both releases the coins a failed send stranded
165
+ * AND keeps the ledger from growing over the wallet's lifetime.
166
+ *
167
+ * CALL THIS ONLY AFTER A BROADCAST FAILED. On the success path the node is
168
+ * still 10-65ms behind and WOULD list the inputs we just legitimately spent —
169
+ * rebuilding there hands them back to the picker and re-creates the exact
170
+ * double-spend this module exists to prevent. Failure is the only moment we
171
+ * know the node's view is the correct one.
172
+ *
173
+ * Clearing both cases is safe because the picker's pool IS the node's list and
174
+ * this ledger only subtracts from it: a coin the node has dropped cannot be
175
+ * picked whether or not we remember it. The one residual is a coin from an
176
+ * earlier SUCCESSFUL send the node has not caught up on — which is why this is
177
+ * failure-only, and why the failing broadcast's own round-trip (~86ms observed
178
+ * between paired sends, against 10-65ms of node lag) is what covers it.
179
+ *
180
+ * `nodeUtxos` must come from the node, not from any cache.
181
+ */
182
+ export function rebuildFromNode (nodeUtxos) {
183
+ const live = new Set(nodeUtxos.map(u => op(u.txid, u.vout)))
184
+ const m = backend.read()
185
+ let released = 0 // node still lists it -> our send never landed, give it back
186
+ let dropped = 0 // node no longer lists it -> confirmed spent, flag is dead
187
+ for (const k of Object.keys(m)) {
188
+ if (live.has(k)) released++; else dropped++
189
+ delete m[k]
190
+ }
191
+ backend.write(m)
192
+ return { released, dropped }
193
+ }
194
+
195
+ export function ledgerSize () { return Object.keys(backend.read()).length }