libp2r2p 0.10.18 → 0.10.20

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -12,35 +12,95 @@ For remote-work scheduling, see [`libp2r2p/network`](network/README.md):
12
12
  `isOnline` probes connectivity and `onOnline` shares recovery monitoring,
13
13
  including retries when the browser omits its native `online` event.
14
14
 
15
- ## Relay feed lifecycle
16
-
17
- `RelayPool.getEventsFeedGenerator(filter, relays, options)` yields events directly,
18
- including the existing `event.meta.relay` field. By default it combines an initial
19
- historical query with a live subscription. `live: false` selects history only;
20
- `filter.limit: 0` selects live only. `getLiveEventsGenerator` exposes the same
21
- lifecycle controls along with its existing `ready` and `readyRelays` properties.
22
-
23
- The returned iterator has a synchronous, idempotent `stopAndDrain()` method. It
24
- closes subscription input and cancels reconnections and outstanding historical
25
- queries, while retaining events already accepted by the library's receive
26
- callbacks. Continue consuming the iterator to obtain those events and observe
27
- completion. This includes initial history, buffered live events, and both history
28
- and live buffers from reconnect recovery. Calling it before the first `next()`
29
- prevents subscriptions from opening. No reception timestamp or new event metadata
30
- is added. Events rejected by ordinary filtering/deduplication remain excluded.
15
+ ## Relay reads and lifecycle
16
+
17
+ All three `RelayPool` read generators emit typed envelopes. The Nostr event is
18
+ never decorated with `meta`; relay provenance belongs to its enclosing item:
19
+
20
+ ```js
21
+ { type: 'event', event, relay }
22
+ { type: 'error', relay, error }
23
+ { type: 'eose', relays: [{ relay, status, error }] } // error is optional
24
+ ```
25
+
26
+ `eose` means the **initial read attempt is complete**, including empty reads and
27
+ partial results. Its report distinguishes actual relay EOSE from other outcomes:
28
+
29
+ | Status | Meaning |
30
+ | --- | --- |
31
+ | `eose` | The relay sent EOSE. |
32
+ | `satisfied` | The requested limit or IDs were satisfied, so the read closed early. |
33
+ | `timeout` | The initial operation deadline elapsed. Also emits an error item. |
34
+ | `cutoff` | The grace period after the first qualifying EOSE/early completion elapsed. |
35
+ | `closed` | The subscription ended without EOSE or an explicit error. |
36
+ | `error` | Connection or subscription failed. Also emits an error item. |
37
+
38
+ Each normalized relay URL gets one report entry, preserving the first spelling
39
+ provided by the caller. An empty relay list emits `{ type: 'eose', relays: [] }`.
40
+ Caller cancellation never fabricates an initial completion. Individual relay
41
+ failures do not prevent other relays from delivering; malformed arguments and
42
+ general operation failures reject the call or iterator read.
43
+
44
+ `getEvents(filter, relays, options)` resolves to
45
+ `{ result: [{ event, relay }], errors: [{ relay, reason }], success, relays }`.
46
+ Its optional `callback` receives the same event/error/EOSE envelopes immediately.
47
+ `success` retains its existing meaning: an event was received or at least one
48
+ relay completed without an error; it does not mean every relay sent EOSE.
49
+ `getEventsGenerator` emits those envelopes and retains that report as its final
50
+ iterator return value (not visible inside a `for await` loop).
51
+
52
+ `getEventsFeedGenerator(filter, relays, options)` combines history and live
53
+ subscription by default: history items, one historical `eose`, buffered live
54
+ items, then ongoing live delivery. `live: false` ends after history and its marker;
55
+ `filter.limit: 0` skips history and forwards the live stream's initial marker.
56
+ Internal live/reconnect EOSEs do not produce additional feed markers.
57
+
58
+ `getLiveEventsGenerator` discards retained events received before each relay's
59
+ EOSE. A ready relay can deliver live events before the aggregate marker. Initial
60
+ `timeout` defaults to 5000 ms (`null` disables it); expiration reports pending
61
+ relays without stopping their connections or subsequent recovery. The grace
62
+ period `timeoutAfterFirstEose` defaults to 500 ms (`null` disables it). Historical
63
+ queries use these same defaults; their grace period starts only when a relay
64
+ with events EOSEs or satisfies its filter. Reconnect-gap reads keep their separate
65
+ `timeoutForReconnectGap` and `timeoutAfterFirstReconnectGapEose` options.
66
+
67
+ The live iterator retains `ready: Promise<{ relays, errors }>` and the
68
+ `readyRelays` getter. `ready` is the initial readiness snapshot, derived from the
69
+ same outcomes as the marker; its errors use `{ relay, reason }`. `readyRelays`
70
+ tracks currently ready relays and can change after that snapshot. Timeouts and
71
+ cutoffs are not acknowledgements of readiness. Reconnections do not repeat the
72
+ initial marker. Both APIs require consuming the iterator to start its work.
73
+
74
+ Live and feed iterators have a synchronous, idempotent `stopAndDrain()` method.
75
+ It closes subscription input and cancels reconnections and outstanding historical
76
+ queries, while retaining items already accepted by receive callbacks. Continue
77
+ consuming to obtain those items and completion. This includes initial history,
78
+ buffered live events, and reconnect recovery buffers. Calling it before the
79
+ first `next()` prevents subscriptions from opening. It does not synthesize EOSE.
31
80
 
32
81
  Aborting `options.signal`, calling `return()` (including a `for await` break), or
33
- calling `throw()` cancels input and pending delivery instead. An event already
34
- delivered to the caller cannot be recalled. These operations also interrupt a
35
- drain. `stopAndDrain()` does not consume the iterator or return a completion
36
- promise: completion is the iterator's `{ done: true }` result.
82
+ calling `throw()` cancels input and pending delivery instead. An item already
83
+ delivered cannot be recalled. These operations can also interrupt a drain.
84
+ `stopAndDrain()` does not consume the iterator or return a completion promise:
85
+ completion is the iterator's `{ done: true }` result.
37
86
 
38
87
  ```js
39
88
  const stream = relayPool.getEventsFeedGenerator({ authors: [pubkey] }, [relay], { signal })
40
89
  // When this relay is removed, call stream.stopAndDrain() from the list handler.
41
- for await (const event of stream) await store(event)
90
+ for await (const item of stream) {
91
+ if (item.type === 'event') await store(item.event)
92
+ else if (item.type === 'error') reportError(item.relay, item.error)
93
+ else if (item.type === 'eose') initialReadFinished(item.relays)
94
+ }
42
95
  ```
43
96
 
97
+ Migration from 0.10.18: replace raw live/feed event reads with `item.event` after
98
+ checking `item.type`, replace `event.meta.relay` with the envelope's `relay`, and
99
+ unwrap each `getEvents().result` entry. There is no compatibility flag. Query,
100
+ content-key, private-channel and NIP-46 helpers unwrap pool results internally
101
+ and retain their higher-level event contracts. Count, publication and disconnect
102
+ return formats are unchanged.
103
+
44
104
  ## Private Messenger
45
105
 
46
106
  The main API is `createPrivateMessenger` from `libp2r2p/private-messenger`.
@@ -66,8 +126,9 @@ const messenger = await createPrivateMessenger({
66
126
  })
67
127
 
68
128
  async function logMessages () {
69
- for await (const message of messenger.messages()) {
129
+ for await (const { message, ack } of messenger.messages()) {
70
130
  console.log(message.type, message.payload)
131
+ await ack()
71
132
  }
72
133
  }
73
134
 
@@ -146,7 +207,7 @@ if (sent.delivery.deletionSeckey) {
146
207
  const deletion = finalizeEvent({
147
208
  kind: 5,
148
209
  created_at: Math.floor(Date.now() / 1000),
149
- tags: [['k', '3560'], ...outerEvents.slice(offset, offset + 100).map(event => ['e', event.id])],
210
+ tags: [['k', '3560'], ...outerEvents.slice(offset, offset + 100).map(({ event }) => ['e', event.id])],
150
211
  content: ''
151
212
  }, deletionKey.secretKey)
152
213
  await relayPool.sendEvent(deletion, relays)
@@ -256,23 +317,101 @@ needed by private channels. For double-DH content-key use, pass a
256
317
  `contentKeySigner` or a signer implementation that handles content keys
257
318
  internally.
258
319
 
259
- Messages are stored in a bounded, durable IndexedDB queue until consumed or
320
+ Messages are stored in a bounded, durable IndexedDB queue until acknowledged or
260
321
  until the principal identity has been inactive for 60 days:
261
322
 
262
323
  ```js
263
324
  async function handleMessages () {
264
- for await (const message of messenger.messages()) {
265
- if (message.type === 'message') {
266
- console.log(message.payload)
267
- }
325
+ for await (const { message, ack } of messenger.messages()) {
326
+ await persistMessageIdempotently(message)
327
+ await ack() // only after the destination commits
268
328
  }
269
329
  }
270
330
 
271
331
  handleMessages().catch(err => console.warn('private messenger messages failed', err))
272
332
  ```
273
333
 
274
- For one-at-a-time consumption, use `await messenger.nextMessage()`. Queue
275
- clearing is asynchronous too: `await messenger.clearChannel(channelPubkey)`.
334
+ For one-at-a-time consumption, `await messenger.nextMessage()` returns
335
+ `{ message, ack, nack }` or `null` if no record is currently available. `ack()`
336
+ removes the reserved record; `nack()` releases it for another attempt. Both
337
+ return a boolean and are idempotent for the same reservation. An expired or
338
+ superseded reservation returns `false` and cannot delete a later delivery.
339
+
340
+ Delivery leases last 30 seconds and renew every 10 seconds while held. Closing
341
+ the messenger or returning the iterator releases its unacknowledged deliveries;
342
+ returning an iterator also cancels an empty pending read. A crashed/suspended
343
+ consumer's expired leases are reclaimable by another instance. Apps must save
344
+ idempotently: a crash between their commit and `ack()` can repeat delivery.
345
+ The queue is shared by identity, so concurrent consumers claim distinct records.
346
+
347
+ The app-message queue defaults to 16 MiB and rejects writes at capacity, without
348
+ evicting pending messages. Capacity pressure pauses ingestion and records a
349
+ recovery gap; successful acknowledgments resume it when possible. While capacity-paused, a
350
+ one-second capacity check also detects space released by another instance. Oversized
351
+ records and storage failures report errors and pause with reason `storage`;
352
+ after correcting the cause, call `resume('storage')`. Retention cleanup and
353
+ explicit `clearQueue()`/`clearChannel()` remain destructive. The queue's budget
354
+ bounds serialized storage, not the size of every intermediate decoded message.
355
+
356
+ ### Forwarded authors and isolated consumers
357
+
358
+ `broadcastRumor({ rumor, ... })` accepts an optional `rumor.pubkey`. Omission
359
+ means the sender; a different pubkey forwards that claimed author and requires
360
+ the original `created_at`. A supplied ID must match the reconstructed event.
361
+ The wire retains the original fields; recipients never overwrite an explicit
362
+ author with the transport sender. Signed events retain their signature.
363
+
364
+ Received messages expose `senderPubkey` and `provenance` alongside `event`,
365
+ `outer`, `meta` and `payload`. Provenance is `direct`, `hearsay`, or `signed`,
366
+ calculated by the receiver. Pending deduplication includes provenance, so an original can upgrade a queued
367
+ hearsay copy of the same event. Private-channel callbacks also expose these fields
368
+ in their metadata. Nothing is attached to the Nostr event. A rumor naming the
369
+ recipient is still hearsay unless independently matched to a known original.
370
+ Forwarded controls cannot invoke ask/reply/presence/recovery handlers in the
371
+ name of another author. Direct rumors are not transferable signed proof.
372
+
373
+ Each PrivateMessenger owns a `createPrivateMessageSession()` from
374
+ `libp2r2p/private-message`. A session groups relay reads for one receiver and
375
+ owns its watches, callbacks, and scoped receive-fragment progress. Independent
376
+ sessions can watch the same channel, including with the same receiver, without
377
+ overwriting or closing each other's reads. The module-level watch helpers are
378
+ a convenience scope per receiver; use explicit sessions for separate consumers.
379
+ `receivedChunkScope` can also scope lower-level private-channel reads; receiver
380
+ pubkeys are included in fragment keys. Expiration/cleanup still use the shared
381
+ receive-chunk database and budget.
382
+
383
+ ### Pause, resume and recovery
384
+
385
+ `pause(reason)` and `resume(reason)` accept nonempty strings. Reasons compose:
386
+ removing `network` cannot override an application's `vault` pause. Pause stops
387
+ live watches, recovery fetches and presence publishing, but retains channel
388
+ configuration and pending deliveries. Already-started operations can finish;
389
+ failed ingestion remains recoverable. Send attempts while paused reject.
390
+
391
+ `watch(channels)` declares desired subscriptions. `unwatch(channels)` removes
392
+ that intent while retaining configuration and history. Browser `online` only
393
+ clears the internal `network` pause, never restarts explicitly unwatched
394
+ channels. Updating retained channel configuration preserves explicit unwatch;
395
+ new channels are watched unless paused. Applications may explicitly watch again.
396
+
397
+ Pauses, unwatch and close record the start of the gap, even for channels that
398
+ have never received a message. Rewatch/resume uses live delivery plus historical
399
+ recovery with overlap. A failed resume remains retryable under its pause reason.
400
+ Before opening a watch, the messenger persists a pending recovery interval.
401
+ A first watch scans the configured recovery window (seven days by default),
402
+ including messages sent before this identity first opened the channel. Successful
403
+ scans persist `recoveredThrough`, even when empty; later watches resume from the
404
+ latest persisted message or scan checkpoint with overlap. Pending older ranges
405
+ remain until successfully recovered, even if newer live messages arrive.
406
+ This also covers abrupt termination without `close()` and channels with no
407
+ messages. Applications still supply signers and desired channels on reopening;
408
+ storage is scoped by origin and user pubkey, not the transient session ID.
409
+ Failed ingestion or incomplete relay fetch reports never complete the gap.
410
+ `lastSeenAt` tracks messages persisted in the incoming queue, not app commits.
411
+ The default recovery window is seven days; recovery still depends on available
412
+ relay/seeder data, and retention limits cannot guarantee indefinite delivery.
413
+
414
+ Queue clearing remains asynchronous: `await messenger.clearChannel(channelPubkey)`.
276
415
 
277
416
  Use explicit subpath imports for bundle size. The package root re-exports the
278
417
  main messenger API for convenience, but applications that only need one piece
@@ -478,11 +617,11 @@ from `libp2r2p/relay`.
478
617
  `getEvents` and `getEventsGenerator` accept `deduplicateAcrossRelays` (boolean,
479
618
  default `true`). With `false`, a matching event is delivered once per relay,
480
619
  while repeated IDs from the same relay remain suppressed. Each occurrence owns
481
- its `meta.relay`; callbacks still run immediately and deadlines, EOSE handling,
482
- and per-relay filter limits are unchanged. The callback/generator item remains
483
- `{ type: 'event', event, relay }`, and the completed query remains
484
- `{ result, errors, success }`. The option does not extend to the live or feed
485
- generators. Callers that need replication coverage can aggregate the returned
620
+ its envelope's `relay`; callbacks still run immediately and per-relay filter
621
+ limits are unchanged. The callback/generator event item is
622
+ `{ type: 'event', event, relay }`; the completed query is
623
+ `{ result: [{ event, relay }], errors, success, relays }`. The option does not
624
+ extend to the live or feed generators. Callers that need replication coverage can aggregate the returned
486
625
  copies by event ID; missing responses do not prove absence from a relay.
487
626
 
488
627
  Publication errors retain their existing `reason` objects and may expose
@@ -566,3 +705,22 @@ limit, retaining the full URL and exposing decoded `url.nfile` and MIME `url.m`.
566
705
 
567
706
  The NIP-94 extension also carries optional `download` intent; see
568
707
  [nip94/README.md](nip94/README.md#download-intent) for event and inline URL forms.
708
+
709
+ ## Reliable IndexedDB queue reservations
710
+
711
+ `createQueue({ prefix, evictionPolicy: 'reject', maxBytes })` opts out of
712
+ capacity eviction. Oversized items reject with `QUEUE_ITEM_TOO_LARGE`, full
713
+ queues with `QUEUE_CAPACITY_EXCEEDED`; browser quota errors propagate without
714
+ trimming existing records. Other eviction policies retain their cache behavior.
715
+ Opening a reject-policy queue under a smaller budget preserves existing items.
716
+
717
+ `queue.reserve({ leaseMs = 30000 })` returns `null` or
718
+ `{ item, ack, nack, renew }`. Reservation token/deadline are internal record
719
+ metadata, persisted atomically in the same IndexedDB transaction as selection.
720
+ `renew()` extends a still-valid lease. Low-level callers own renewal and release;
721
+ PrivateMessenger supplies both. Do not mix reservations with destructive or
722
+ position-shifting queue operations on the same queue except explicit removal.
723
+ A stale token cannot acknowledge a newly reserved or replaced record.
724
+
725
+ `queue.getCapacity()` returns `{ usedBytes, maxBytes }`; capacity rejections
726
+ also expose `requiredBytes` and `maxBytes`, without retaining the rejected item.
@@ -103,7 +103,7 @@ export async function getIykcProofs (pubkeys, {
103
103
  timeout: 5000,
104
104
  timeoutAfterFirstEose: null
105
105
  })
106
- return result
106
+ return result.map(({ event }) => event)
107
107
  })
108
108
  )
109
109
 
@@ -56,6 +56,7 @@ function byteLength (value) {
56
56
 
57
57
  function normalizeEvictionPolicy (policy) {
58
58
  if (policy === 'opposite-end' || policy === undefined || policy === null) return 'opposite-end'
59
+ if (policy === 'reject') return 'reject'
59
60
  if (policy === 'fifo' || policy === 'head') return 'head'
60
61
  if (policy === 'lifo' || policy === 'tail') return 'tail'
61
62
  throw new ValidationError('QUEUE_INVALID_EVICTION_POLICY')
@@ -468,6 +469,12 @@ export async function createQueue ({
468
469
  async function evictToFit (tx, state, requiredBytes, options = {}) {
469
470
  if (!hasByteLimit()) return
470
471
  if (requiredBytes > sessionMaxBytes) throw new Error('QUEUE_ITEM_TOO_LARGE')
472
+ if (configuredEvictionPolicy === 'reject') {
473
+ if (state.usedBytes + requiredBytes > sessionMaxBytes) {
474
+ throw Object.assign(new Error('QUEUE_CAPACITY_EXCEEDED'), { requiredBytes, maxBytes: sessionMaxBytes })
475
+ }
476
+ return
477
+ }
471
478
  // Make room for the write and retain eviction headroom for the next one.
472
479
  const targetBytes = targetBytesAfterWrite(requiredBytes)
473
480
  while (state.usedBytes + requiredBytes > targetBytes) {
@@ -477,7 +484,7 @@ export async function createQueue ({
477
484
  }
478
485
 
479
486
  async function evictToBytes (tx, state, targetBytes, options = {}) {
480
- if (!hasByteLimit()) return
487
+ if (!hasByteLimit() || configuredEvictionPolicy === 'reject') return
481
488
  while (state.usedBytes > targetBytes) {
482
489
  if (!await evictOne(tx, state, options)) break
483
490
  }
@@ -590,7 +597,7 @@ export async function createQueue ({
590
597
  if (wakeWaiters) wake()
591
598
  return value
592
599
  } catch (err) {
593
- if (retried || !hasByteLimit() || !isQuotaExceeded(err)) throw err
600
+ if (configuredEvictionPolicy === 'reject' || retried || !hasByteLimit() || !isQuotaExceeded(err)) throw err
594
601
  // Browser quota can be lower than the configured logical budget.
595
602
  // Retry once with a smaller in-memory budget after atomic rollback.
596
603
  lowerSessionMaxBytes(requiredBytes)
@@ -748,6 +755,61 @@ export async function createQueue ({
748
755
  })
749
756
  }
750
757
 
758
+ // Reservations are stored atomically with the item; another tab/process can
759
+ // reclaim an expired lease after a crash. Destructive/positional queue APIs
760
+ // must not be mixed with reservations on the same queue.
761
+ async function reserve ({ leaseMs = 30000, now = Date.now() } = {}) {
762
+ if (!Number.isSafeInteger(leaseMs) || leaseMs <= 0 || !Number.isSafeInteger(now) || now < 0) throw new ValidationError('QUEUE_INVALID_LEASE')
763
+ const token = globalThis.crypto.randomUUID()
764
+ const reserved = await mutate(async tx => {
765
+ const p = deferred()
766
+ let cursor = (await run('openCursor', [], ITEMS_STORE, null, { tx, p })).result
767
+ while (cursor) {
768
+ const record = cursor.value
769
+ if (!record.reservation || record.reservation.until <= now) {
770
+ record.reservation = { token, until: now + leaseMs }
771
+ await putRecord(tx, record)
772
+ return record
773
+ }
774
+ cursor = await nextCursor(cursor, p)
775
+ }
776
+ return null
777
+ })
778
+ if (!reserved) return null
779
+ let settled = false
780
+ let completedAction
781
+ let completedResult
782
+ let tail = Promise.resolve()
783
+ const update = (action, at = Date.now()) => {
784
+ const operation = tail.then(async () => {
785
+ if (settled) return action === completedAction ? completedResult : false
786
+ const result = await mutate(async (tx, state) => {
787
+ const record = await getRecord(tx, reserved.position)
788
+ if (record?.reservation?.token !== token || record.reservation.until <= at) return false
789
+ if (action === 'ack') {
790
+ await deleteRecord(tx, record.position)
791
+ state.usedBytes = Math.max(0, state.usedBytes - record.byteSize)
792
+ await trimBounds(tx, state)
793
+ } else {
794
+ if (action === 'renew') record.reservation.until = at + leaseMs
795
+ else delete record.reservation
796
+ await putRecord(tx, record)
797
+ }
798
+ return true
799
+ }, { wakeWaiters: action !== 'renew' })
800
+ if (action !== 'renew' || !result) {
801
+ settled = true
802
+ completedAction = action
803
+ completedResult = result
804
+ }
805
+ return result
806
+ })
807
+ tail = operation.catch(() => {})
808
+ return operation
809
+ }
810
+ return { item: reserved.item, ack: () => update('ack'), nack: () => update('nack'), renew: () => update('renew') }
811
+ }
812
+
751
813
  async function snapshotStoredItems () {
752
814
  return snapshot(async tx => {
753
815
  const state = await readState(tx)
@@ -819,6 +881,8 @@ export async function createQueue ({
819
881
  })
820
882
 
821
883
  return {
884
+ reserve,
885
+ getCapacity: () => snapshot(async tx => ({ usedBytes: (await readState(tx)).usedBytes, maxBytes: sessionMaxBytes })),
822
886
  enqueue: push,
823
887
  push,
824
888
  pop,
@@ -75,12 +75,16 @@ export class Nip46Client {
75
75
  limit: 0
76
76
  }, parsed.relays, {
77
77
  signal: controller.signal,
78
+ timeout: options.timeout ?? DEFAULT_TIMEOUT,
78
79
  timeoutAfterFirstEose: options.timeoutAfterFirstEose ?? DEFAULT_TIMEOUT_AFTER_FIRST_EOSE
79
80
  })
80
81
  const found = Promise.withResolvers()
81
82
  const consume = (async () => {
82
83
  try {
83
- for await (const event of stream) {
84
+ for await (const item of stream) {
85
+ if (item.type === 'error') { options.onError?.(item.error); continue }
86
+ if (item.type !== 'event') continue
87
+ const { event } = item
84
88
  if (!isNip46EventFor(event, clientPubkey)) continue
85
89
  const response = decodeNip46Frame(event, clientSecretKey)
86
90
  if (response?.result === parsed.secret) {
@@ -95,14 +95,16 @@ export class Nip46Transport {
95
95
  const controller = new AbortController()
96
96
  const stream = this.#relayPool.getLiveEventsGenerator(filter, relays, {
97
97
  signal: controller.signal,
98
+ timeout: this.#operationTimeout,
98
99
  timeoutAfterFirstEose: this.#timeoutAfterFirstEose
99
100
  })
100
101
  const context = { controller, stream, relays: [...relays], consume: null }
101
102
  this.#contexts.add(context)
102
103
  context.consume = (async () => {
103
104
  try {
104
- for await (const event of stream) {
105
- Promise.resolve(onEvent(event)).catch(error => this.#reportError(error))
105
+ for await (const item of stream) {
106
+ if (item.type === 'error') this.#reportError(item.error)
107
+ else if (item.type === 'event') Promise.resolve(onEvent(item.event)).catch(error => this.#reportError(error))
106
108
  }
107
109
  } catch (error) {
108
110
  if (!this.#closed && error?.message !== 'Aborted') this.#reportError(error)
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "libp2r2p",
3
- "version": "0.10.18",
3
+ "version": "0.10.20",
4
4
  "description": "Peer-to-relay-to-peer",
5
5
  "keywords": [
6
6
  "p2r2p",
@@ -0,0 +1,26 @@
1
+ import { getEventHash, isSerializableEvent } from '../../event/index.js'
2
+ import { ValidationError } from '../../error/index.js'
3
+
4
+ // Transport sender and claimed author are distinct. This also validates a
5
+ // supplied ID before discarding it from the unsigned wire representation.
6
+ export function normalizeRumor (rumor, senderPubkey) {
7
+ if (!rumor || typeof rumor !== 'object' || Array.isArray(rumor) || 'sig' in rumor) throw new ValidationError('INVALID_RUMOR')
8
+ const event = {
9
+ pubkey: rumor.pubkey === undefined ? senderPubkey : rumor.pubkey,
10
+ kind: rumor.kind,
11
+ created_at: rumor.created_at,
12
+ tags: rumor.tags,
13
+ content: rumor.content
14
+ }
15
+ if (!isSerializableEvent(event)) throw new ValidationError('INVALID_RUMOR')
16
+ const id = getEventHash(event)
17
+ if (rumor.id !== undefined && rumor.id !== id) throw new ValidationError('INVALID_RUMOR_ID')
18
+ return { ...event, id }
19
+ }
20
+
21
+ export function deliveryInfo (event, senderPubkey) {
22
+ return {
23
+ senderPubkey,
24
+ provenance: event.sig ? 'signed' : event.pubkey === senderPubkey ? 'direct' : 'hearsay'
25
+ }
26
+ }
@@ -5,6 +5,7 @@ import { hexToBytes } from '../base16/index.js'
5
5
  import { isValidContentKeyProof, isValidIykcProof, makeContentKeyEventForPubkey, parseContentKeyEvent } from '../content-key/event/index.js'
6
6
  import { getIykcProofs } from '../content-key/index.js'
7
7
  import { ValidationError } from '../error/index.js'
8
+ import { normalizeRumor, deliveryInfo } from './helpers/rumor.js'
8
9
  import * as nip44v3 from '../nip44-v3/index.js'
9
10
  import { relayPool } from '../relay/index.js'
10
11
  import { JSONL_CHUNK_BYTES, NYM_CARRIER_CHUNK_CHARS } from './helpers/chunk-size.js'
@@ -371,8 +372,7 @@ function eventFromPayload ({ payloadCiphertext, messageSeckey, senderPubkey }) {
371
372
  const messagePubkey = getPublicKey(messageSecretKey)
372
373
  const decrypted = JSON.parse(nip44v3.decrypt(messageSecretKey, messagePubkey, ROUTER_KIND, NIP44_V3_SCOPE, payloadCiphertext))
373
374
  if (hasEventSignature(decrypted)) return assertValidSignedInnerEvent(decrypted)
374
- const normalized = { ...decrypted, pubkey: senderPubkey }
375
- return { ...normalized, id: getEventHash(normalized) }
375
+ return normalizeRumor(decrypted, senderPubkey)
376
376
  }
377
377
 
378
378
  async function unwrapRecipientEnvelope ({ payloadCiphertext, envelope, receiverSigner, receiverPubkey, senderPubkey, imkcPubkey, rowScope = '' }) {
@@ -670,6 +670,7 @@ function createProcessor ({
670
670
  onSeedEvent,
671
671
  onContentKeyUsage,
672
672
  onError,
673
+ receivedChunkScope = '',
673
674
  receivedChunkTtlMs = DEFAULT_RECEIVED_CHUNK_TTL_MS,
674
675
  receivedChunkTtlMsByPubkey,
675
676
  receivedChunkMaxBytes = DEFAULT_RECEIVED_CHUNK_MAX_BYTES,
@@ -678,6 +679,7 @@ function createProcessor ({
678
679
  ignoredGroupMaxEntries = DEFAULT_IGNORED_GROUP_MAX_ENTRIES
679
680
  }) {
680
681
  const receivedChunks = createReceivedChunkStore({
682
+ scope: `${receivedChunkScope}:${receiverPubkey || ''}`,
681
683
  ttlMs: receivedChunkTtlMs,
682
684
  maxBytes: receivedChunkMaxBytes,
683
685
  indexedDB: receivedChunkIndexedDB
@@ -750,7 +752,8 @@ function createProcessor ({
750
752
  await onNymEvent?.(event, outer, {
751
753
  carrier: carriers[0],
752
754
  carriers,
753
- channelPubkey
755
+ channelPubkey,
756
+ ...deliveryInfo(event, carriers[0].pubkey)
754
757
  })
755
758
  await receivedChunks.removeGroup(groupKey)
756
759
  return
@@ -833,7 +836,7 @@ function createProcessor ({
833
836
  })
834
837
 
835
838
  if (event && !mustScanWholeBundle) {
836
- await onEvent?.(event, outer, { router: joinedRouter(router), channelPubkey })
839
+ await onEvent?.(event, outer, { router: joinedRouter(router), channelPubkey, ...deliveryInfo(event, senderPubkey) })
837
840
  ignoredGroups.add(groupKey)
838
841
  await receivedChunks.removeGroup(groupKey)
839
842
  return
@@ -855,7 +858,7 @@ function createProcessor ({
855
858
  onContentKeyUsage
856
859
  })
857
860
  if (shouldSeed) await onSeedEvent?.({ recordType: 'routerRow_v1', outer, router: completeRouter, channelPubkey, jsonl, innerEventIdsByRowIndex })
858
- if (event) await onEvent?.(event, outer, { router: completeRouter, channelPubkey, jsonl })
861
+ if (event) await onEvent?.(event, outer, { router: completeRouter, channelPubkey, jsonl, ...deliveryInfo(event, senderPubkey) })
859
862
 
860
863
  await receivedChunks.removeGroup(groupKey)
861
864
  } catch (err) {
@@ -863,6 +866,7 @@ function createProcessor ({
863
866
  ignoredGroups.add(groupKey)
864
867
  await receivedChunks.removeGroup(groupKey).catch(() => {})
865
868
  }
869
+ if (groupKey && !shouldIgnoreGroupError(err)) await receivedChunks.removeGroup(groupKey).catch(() => {})
866
870
  onError?.(err)
867
871
  }
868
872
  }
@@ -894,7 +898,7 @@ function shouldIgnoreGroupError (err) {
894
898
  ].includes(err?.message)
895
899
  }
896
900
 
897
- export async function fetch ({ receiverSigner, iykcSigner, privateChannelSigner = receiverSigner, privateChannelSignersByPubkey, privateChannelReaderSigner = privateChannelSigner, privateChannelReaderSignersByPubkey, privateChannelReaderPubkey, privateChannelReaderPubkeysByPubkey, privateChannelPubkey, privateChannelPubkeys, receiverPubkey, relays, onChunk, onEvent, onNymEvent, onSeedEvent, onContentKeyUsage, onError, since, until, limit, mode = 'leecher', modeByPubkey, receivedChunkTtlMs = DEFAULT_RECEIVED_CHUNK_TTL_MS, receivedChunkTtlMsByPubkey, receivedChunkMaxBytes = DEFAULT_RECEIVED_CHUNK_MAX_BYTES, receivedChunkIndexedDB = globalThis.indexedDB, ignoredGroupTtlMs = DEFAULT_IGNORED_GROUP_TTL_MS, ignoredGroupMaxEntries = DEFAULT_IGNORED_GROUP_MAX_ENTRIES, _getEvents = getEvents }) {
901
+ export async function fetch ({ signal, receiverSigner, iykcSigner, privateChannelSigner = receiverSigner, privateChannelSignersByPubkey, privateChannelReaderSigner = privateChannelSigner, privateChannelReaderSignersByPubkey, privateChannelReaderPubkey, privateChannelReaderPubkeysByPubkey, privateChannelPubkey, privateChannelPubkeys, receiverPubkey, relays, onChunk, onEvent, onNymEvent, onSeedEvent, onContentKeyUsage, onError, since, until, limit, mode = 'leecher', modeByPubkey, receivedChunkScope = '', receivedChunkTtlMs = DEFAULT_RECEIVED_CHUNK_TTL_MS, receivedChunkTtlMsByPubkey, receivedChunkMaxBytes = DEFAULT_RECEIVED_CHUNK_MAX_BYTES, receivedChunkIndexedDB = globalThis.indexedDB, ignoredGroupTtlMs = DEFAULT_IGNORED_GROUP_TTL_MS, ignoredGroupMaxEntries = DEFAULT_IGNORED_GROUP_MAX_ENTRIES, _getEvents = getEvents }) {
898
902
  if (!relays?.length) throw new ValidationError('NO_RELAYS')
899
903
  const authors = privateChannelPubkeyList({ privateChannelPubkey, privateChannelPubkeys })
900
904
  const filter = { kinds: [PRIVATE_BROADCAST_KIND] }
@@ -903,21 +907,30 @@ export async function fetch ({ receiverSigner, iykcSigner, privateChannelSigner
903
907
  if (until != null) filter.until = until
904
908
  if (limit != null) filter.limit = limit
905
909
 
906
- const { result: events } = await _getEvents(filter, relays, {
910
+ const { result, errors = [], relays: report = [] } = await _getEvents(filter, relays, {
907
911
  timeout: 5000,
908
- timeoutAfterFirstEose: null
912
+ timeoutAfterFirstEose: null,
913
+ ...(signal ? { signal } : {})
909
914
  })
915
+ const events = result.map(({ event }) => event)
910
916
  events.sort((a, b) => a.created_at - b.created_at)
911
- const processOuterEvent = createProcessor({ receiverSigner, iykcSigner, privateChannelSigner, privateChannelSignersByPubkey, privateChannelReaderSigner, privateChannelReaderSignersByPubkey, privateChannelReaderPubkey, privateChannelReaderPubkeysByPubkey, receiverPubkey, mode, modeByPubkey, onChunk, onEvent, onNymEvent, onSeedEvent, onContentKeyUsage, onError, receivedChunkTtlMs, receivedChunkTtlMsByPubkey, receivedChunkMaxBytes, receivedChunkIndexedDB, ignoredGroupTtlMs, ignoredGroupMaxEntries })
917
+ const processOuterEvent = createProcessor({ receiverSigner, iykcSigner, privateChannelSigner, privateChannelSignersByPubkey, privateChannelReaderSigner, privateChannelReaderSignersByPubkey, privateChannelReaderPubkey, privateChannelReaderPubkeysByPubkey, receiverPubkey, mode, modeByPubkey, onChunk, onEvent, onNymEvent, onSeedEvent, onContentKeyUsage, onError, receivedChunkScope, receivedChunkTtlMs, receivedChunkTtlMsByPubkey, receivedChunkMaxBytes, receivedChunkIndexedDB, ignoredGroupTtlMs, ignoredGroupMaxEntries })
912
918
  try {
913
- for (const event of events) await processOuterEvent(event)
919
+ for (const event of events) {
920
+ signal?.throwIfAborted()
921
+ await processOuterEvent(event)
922
+ }
923
+ signal?.throwIfAborted()
924
+ if (errors.length || report.some(entry => !['eose', 'satisfied'].includes(entry.status))) {
925
+ throw new AggregateError(errors.map(entry => entry.reason), 'PRIVATE_CHANNEL_FETCH_INCOMPLETE')
926
+ }
914
927
  return events
915
928
  } finally {
916
929
  processOuterEvent.close()
917
930
  }
918
931
  }
919
932
 
920
- export function subscribe ({ receiverSigner, iykcSigner, privateChannelSigner = receiverSigner, privateChannelSignersByPubkey, privateChannelReaderSigner = privateChannelSigner, privateChannelReaderSignersByPubkey, privateChannelReaderPubkey, privateChannelReaderPubkeysByPubkey, privateChannelPubkey, privateChannelPubkeys, receiverPubkey, relays, onChunk, onEvent, onNymEvent, onSeedEvent, onContentKeyUsage, onError, since = nowSeconds() - 5, limit, liveOnly = false, mode = 'leecher', modeByPubkey, receivedChunkTtlMs = DEFAULT_RECEIVED_CHUNK_TTL_MS, receivedChunkTtlMsByPubkey, receivedChunkMaxBytes = DEFAULT_RECEIVED_CHUNK_MAX_BYTES, receivedChunkIndexedDB = globalThis.indexedDB, ignoredGroupTtlMs = DEFAULT_IGNORED_GROUP_TTL_MS, ignoredGroupMaxEntries = DEFAULT_IGNORED_GROUP_MAX_ENTRIES, _liveEventsGenerator = getLiveEventsGenerator, _eventsFeedGenerator = getEventsFeedGenerator }) {
933
+ export function subscribe ({ receiverSigner, iykcSigner, privateChannelSigner = receiverSigner, privateChannelSignersByPubkey, privateChannelReaderSigner = privateChannelSigner, privateChannelReaderSignersByPubkey, privateChannelReaderPubkey, privateChannelReaderPubkeysByPubkey, privateChannelPubkey, privateChannelPubkeys, receiverPubkey, relays, onChunk, onEvent, onNymEvent, onSeedEvent, onContentKeyUsage, onError, since = nowSeconds() - 5, limit, liveOnly = false, mode = 'leecher', modeByPubkey, receivedChunkScope = '', receivedChunkTtlMs = DEFAULT_RECEIVED_CHUNK_TTL_MS, receivedChunkTtlMsByPubkey, receivedChunkMaxBytes = DEFAULT_RECEIVED_CHUNK_MAX_BYTES, receivedChunkIndexedDB = globalThis.indexedDB, ignoredGroupTtlMs = DEFAULT_IGNORED_GROUP_TTL_MS, ignoredGroupMaxEntries = DEFAULT_IGNORED_GROUP_MAX_ENTRIES, _liveEventsGenerator = getLiveEventsGenerator, _eventsFeedGenerator = getEventsFeedGenerator }) {
921
934
  if (!relays?.length) throw new ValidationError('NO_RELAYS')
922
935
  if (receiverSigner && !receiverSigner?.nip44DecryptDoubleDH && !receiverSigner?.nip44v3Decrypt) throw new ValidationError('RECEIVER_SIGNER_NIP44V3_DECRYPT_UNSUPPORTED')
923
936
  if (!privateChannelReaderSigner && !privateChannelReaderSignersByPubkey && !privateChannelSigner && !privateChannelSignersByPubkey) throw new ValidationError('PRIVATE_CHANNEL_READER_REQUIRED')
@@ -926,7 +939,7 @@ export function subscribe ({ receiverSigner, iykcSigner, privateChannelSigner =
926
939
  const filter = { kinds: [PRIVATE_BROADCAST_KIND], since }
927
940
  if (authors.length) filter.authors = authors
928
941
  if (limit != null) filter.limit = limit
929
- const processOuterEvent = createProcessor({ receiverSigner, iykcSigner, privateChannelSigner, privateChannelSignersByPubkey, privateChannelReaderSigner, privateChannelReaderSignersByPubkey, privateChannelReaderPubkey, privateChannelReaderPubkeysByPubkey, receiverPubkey, mode, modeByPubkey, onChunk, onEvent, onNymEvent, onSeedEvent, onContentKeyUsage, onError, receivedChunkTtlMs, receivedChunkTtlMsByPubkey, receivedChunkMaxBytes, receivedChunkIndexedDB, ignoredGroupTtlMs, ignoredGroupMaxEntries })
942
+ const processOuterEvent = createProcessor({ receiverSigner, iykcSigner, privateChannelSigner, privateChannelSignersByPubkey, privateChannelReaderSigner, privateChannelReaderSignersByPubkey, privateChannelReaderPubkey, privateChannelReaderPubkeysByPubkey, receiverPubkey, mode, modeByPubkey, onChunk, onEvent, onNymEvent, onSeedEvent, onContentKeyUsage, onError, receivedChunkScope, receivedChunkTtlMs, receivedChunkTtlMsByPubkey, receivedChunkMaxBytes, receivedChunkIndexedDB, ignoredGroupTtlMs, ignoredGroupMaxEntries })
930
943
  const controller = new AbortController()
931
944
  const events = liveOnly
932
945
  ? _liveEventsGenerator(filter, relays, {
@@ -940,9 +953,10 @@ export function subscribe ({ receiverSigner, iykcSigner, privateChannelSigner =
940
953
 
941
954
  async function consumeEvents () {
942
955
  try {
943
- for await (const outer of events) {
956
+ for await (const item of events) {
944
957
  if (controller.signal.aborted) continue
945
- await processOuterEvent(outer)
958
+ if (item.type === 'error') onError?.(item.error)
959
+ else if (item.type === 'event') await processOuterEvent(item.event)
946
960
  }
947
961
  } catch (error) {
948
962
  if (!controller.signal.aborted && error?.message !== 'Aborted') onError?.(error)
@@ -20,7 +20,7 @@ IndexedDB schema, scoped per received-chunk prefix:
20
20
  database `${prefix}:idb`, version 2
21
21
 
22
22
  groups, keyPath "groupKey"
23
- groupKey `${channelPubkey}:${routerPubkey}`
23
+ groupKey optional consumer scope + `${channelPubkey}:${routerPubkey}`
24
24
  channelPubkey/routerPubkey private-channel coordinates
25
25
  total expected chunk count
26
26
  received/receivedCount sparse received-index map and its count
@@ -181,6 +181,7 @@ function wait (ms) {
181
181
 
182
182
  export function createReceivedChunkStore ({
183
183
  prefix = DEFAULT_PREFIX,
184
+ scope = '',
184
185
  indexedDB = globalThis.indexedDB,
185
186
  ttlMs = DEFAULT_RECEIVED_CHUNK_TTL_MS,
186
187
  maxBytes = DEFAULT_RECEIVED_CHUNK_MAX_BYTES
@@ -220,7 +221,7 @@ export function createReceivedChunkStore ({
220
221
  }
221
222
 
222
223
  function groupKeyFor (channelPubkey, routerPubkey) {
223
- return `${channelPubkey}:${routerPubkey}`
224
+ return `${scope ? `${scope}:` : ''}${channelPubkey}:${routerPubkey}`
224
225
  }
225
226
 
226
227
  async function readUsage (tx) {