libp2r2p 0.11.4 → 0.11.6

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -534,6 +534,42 @@ returned events; received messages may already have been queued. This is a
534
534
  history completeness failure, not evidence that an outgoing message failed.
535
535
  Existing rewatch/resume and relay-list refresh paths retry pending intervals;
536
536
  there is no dedicated periodic retry timer for a failed history fetch.
537
+ `private-channel.fetchHistory(options)` (0.11.6) is the complete-interval API
538
+ used by messenger recovery. It accepts the signers, processor/fragment options,
539
+ callbacks and abort signal of `fetch`, plus inclusive `since`/`until` bounds
540
+ (defaults: 0 and the start time of the call). It returns only
541
+ `{ oldestCreatedAt, receivedEventCount, relays }`; an empty scan has a null oldest
542
+ time. No complete-history event array is retained. `fetch` keeps its existing
543
+ single-query array contract and optional limit.
544
+
545
+ History queries each relay independently with limit 16. A response with 16 events
546
+ or status `satisfied` is saturated, even if it already reports EOSE or contains
547
+ fewer valid events. Saturated intervals split into nonoverlapping inclusive
548
+ halves; completed pages process chronologically, oldest half first. Single-second
549
+ intervals grow through 16, 32, 64, 128 and 256. Each response is also bounded to
550
+ 4 MiB of serialized UTF-8 event data. At the dense-second or volume ceiling,
551
+ `PRIVATE_CHANNEL_FETCH_INCOMPLETE` retains the operational cause
552
+ `PRIVATE_CHANNEL_HISTORY_PAGE_LIMIT`; the interval stays pending. There is no
553
+ nonstandard cursor, disk spool, dropped excess or successful incomplete scan.
554
+
555
+ A fragment processor and the existing scoped fragment storage span the entire
556
+ call, including pages and relays. Callbacks are awaited before requesting the
557
+ next page. Processing/persistence failures propagate; completed pages may already
558
+ be ingested and retries remain subject to the existing idempotence contract.
559
+ The summary count includes processed outer events per relay, including replicas,
560
+ not unique decrypted messages or discarded saturated probes. It does not include
561
+ all events observed in those probes.
562
+
563
+ Recovery admits at most two pages per PrivateMessenger and one per normalized
564
+ relay across the same library instance, including page processing time. Continuous
565
+ live watches are outside those recovery gates. Waiters are abortable and FIFO,
566
+ with at most 256 queued per gate (`PRIVATE_CHANNEL_HISTORY_QUEUE_FULL` on excess).
567
+ The five-second network timeout begins after admission; reported read time also
568
+ includes admission but excludes processing. Seeders still supplement the oldest
569
+ relay result. Checkpoints advance only after the entire interval and existing
570
+ seeder stages succeed; interruption preserves its original pending range. No
571
+ persisted state format changes or migrations are needed.
572
+
537
573
  Subscription relay errors forwarded to `onError` include `relay` when known
538
574
  and `operation: 'private-channel.subscribe'`. A per-notification wrapper retains
539
575
  the native message, name, code, category and WebSocket close fields; `cause`
@@ -915,10 +951,16 @@ local defaults, including a bounded streaming file cache. The optional
915
951
  provides reusable DM inbox/outbox and attachment lifecycle without UI coupling,
916
952
  including an optional `onSendError` callback identifying the failed outbox message.
917
953
 
918
- Single-recipient private messages now retry NIP-65 read relays in pairs when
954
+ Private messages, including multi-recipient channels, retry NIP-65 read relays
955
+ in pairs per recipient when
919
956
  selected relays reject publication or become unreachable while connectivity is
920
- confirmed. The exact encrypted outer events are reused; explicit per-call and
921
- channel relay overrides remain authoritative. Successful sends keep first-ACK
957
+ confirmed. The exact encrypted outer events are reused. Optional `fallbackRelays` on
958
+ `createPrivateMessenger` and `createPrivateMessageSession` are tried after the
959
+ primary destinations, including explicit per-call `relays`, channel `sendRelays`
960
+ and channel/global `relays`, as well as explicit `relayToReceivers` maps.
961
+ Retries preserve each encrypted recipient subset; success requires accepted
962
+ publication covering every recipient in that subset. Receive routing adds them from the start for live
963
+ messages, history and files while preserving automatic NIP-65 updates. Successful sends keep first-ACK
922
964
  latency, and short-lived channel/recipient preferences avoid recently refused
923
965
  relays without changing receive subscriptions. See the
924
966
  [session coordinator](private-messenger/session/README.md) for deferred offline
@@ -0,0 +1,41 @@
1
+ // A bounded FIFO for asynchronous work. Acquired leases release explicitly;
2
+ // queued cancellations never consume capacity or keep idle keyed gates alive.
3
+ export function createAbortableSemaphore (capacity, onIdle = () => {}) {
4
+ const queue = []
5
+ let active = 0
6
+ const drain = () => {
7
+ while (active < capacity && queue.length) {
8
+ const job = queue.shift()
9
+ job.signal?.removeEventListener('abort', job.abort)
10
+ active++
11
+ let released = false
12
+ job.resolve(() => {
13
+ if (released) return
14
+ released = true
15
+ active--
16
+ drain()
17
+ })
18
+ }
19
+ if (!active && !queue.length) onIdle()
20
+ }
21
+ return {
22
+ acquire (signal) {
23
+ signal?.throwIfAborted()
24
+ if (queue.length >= 256) return Promise.reject(Object.assign(new Error('PRIVATE_CHANNEL_HISTORY_QUEUE_FULL'), { code: 'PRIVATE_CHANNEL_HISTORY_QUEUE_FULL' }))
25
+ return new Promise((resolve, reject) => {
26
+ const job = {
27
+ signal, resolve, abort: () => {
28
+ const index = queue.indexOf(job)
29
+ if (index < 0) return
30
+ queue.splice(index, 1)
31
+ reject(signal.reason)
32
+ drain()
33
+ }
34
+ }
35
+ signal?.addEventListener('abort', job.abort, { once: true })
36
+ queue.push(job)
37
+ drain()
38
+ })
39
+ }
40
+ }
41
+ }
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "libp2r2p",
3
- "version": "0.11.4",
3
+ "version": "0.11.6",
4
4
  "description": "Peer-to-relay-to-peer",
5
5
  "keywords": [
6
6
  "p2r2p",
@@ -22,6 +22,7 @@
22
22
  "ecdh",
23
23
  "error",
24
24
  "event",
25
+ "helpers",
25
26
  "idb",
26
27
  "idb-queue",
27
28
  "index.js",
@@ -0,0 +1,121 @@
1
+ import { ValidationError } from '../../error/index.js'
2
+ import { normalizeRelayUrl } from '../../url/index.js'
3
+ import { createAbortableSemaphore } from '../../helpers/abortable-semaphore.js'
4
+ import { incompleteFetchError } from './fetch-error.js'
5
+
6
+ const PAGE_SIZE = 16
7
+ const MAX_PAGE_SIZE = 256
8
+ const MAX_PAGE_BYTES = 4 * 1024 * 1024
9
+ const TIMEOUT_MS = 5000
10
+ const relayGates = new Map()
11
+ const encoder = new TextEncoder()
12
+ const pageLimitError = () => Object.assign(new Error('PRIVATE_CHANNEL_HISTORY_PAGE_LIMIT'), { code: 'PRIVATE_CHANNEL_HISTORY_PAGE_LIMIT' })
13
+
14
+ function acquireRelay (relay, signal) {
15
+ signal?.throwIfAborted()
16
+ let gate = relayGates.get(relay)
17
+ if (!gate) {
18
+ gate = createAbortableSemaphore(1, () => { if (relayGates.get(relay) === gate) relayGates.delete(relay) })
19
+ relayGates.set(relay, gate)
20
+ }
21
+ return gate.acquire(signal)
22
+ }
23
+
24
+ // Queries bounded temporal leaves, oldest first. A saturated one-second leaf
25
+ // cannot be split using standard NIP-01; expand it only up to the safe ceiling.
26
+ export async function readHistory ({ filter, relays, receiverPubkey, signal, getEvents, processEvent, acquirePage }) {
27
+ const since = filter.since ?? 0
28
+ const until = filter.until ?? Math.floor(Date.now() / 1000)
29
+ if (![since, until].every(value => Number.isSafeInteger(value) && value >= 0) || since > until) throw new ValidationError('INVALID_PRIVATE_CHANNEL_HISTORY_RANGE')
30
+ if (!Array.isArray(relays) || !relays.length) throw new ValidationError('NO_RELAYS')
31
+ const urls = [...new Set(relays.map(normalizeRelayUrl))]
32
+ const report = []
33
+ const errors = []
34
+ let oldestCreatedAt = null
35
+ let receivedEventCount = 0
36
+ let elapsedMs = 0
37
+ const process = async result => {
38
+ result.sort((a, b) => a.event.created_at - b.event.created_at || String(a.event.id).localeCompare(String(b.event.id)))
39
+ for (const { event } of result) {
40
+ signal?.throwIfAborted()
41
+ await processEvent(event)
42
+ receivedEventCount++
43
+ oldestCreatedAt = Math.min(oldestCreatedAt ?? event.created_at, event.created_at)
44
+ }
45
+ }
46
+ for (const relay of urls) {
47
+ const intervals = [{ since, until, limit: PAGE_SIZE }]
48
+ let outcome = { relay, status: 'eose' }
49
+ while (intervals.length) {
50
+ signal?.throwIfAborted()
51
+ const interval = intervals.pop()
52
+ const started = performance.now()
53
+ let releaseRelay
54
+ let releasePage
55
+ let page
56
+ try {
57
+ let bytes = 0
58
+ try {
59
+ releaseRelay = await acquireRelay(relay, signal)
60
+ releasePage = await acquirePage?.(signal)
61
+ signal?.throwIfAborted()
62
+ page = await getEvents({ ...filter, ...interval }, [relay], {
63
+ signal, timeout: TIMEOUT_MS, timeoutAfterFirstEose: null,
64
+ callback: item => {
65
+ if (item.type !== 'event') return
66
+ bytes += encoder.encode(JSON.stringify(item.event)).byteLength
67
+ if (bytes > MAX_PAGE_BYTES) throw pageLimitError()
68
+ }
69
+ })
70
+ // Also enforce the contract for injected readers that omit callbacks.
71
+ let resultBytes = 0
72
+ for (const { event } of page.result) {
73
+ resultBytes += encoder.encode(JSON.stringify(event)).byteLength
74
+ if (resultBytes > MAX_PAGE_BYTES) throw pageLimitError()
75
+ }
76
+ } catch (error) {
77
+ signal?.throwIfAborted()
78
+ errors.push({ relay, reason: error })
79
+ outcome = { relay, status: 'error', error }
80
+ break
81
+ } finally { elapsedMs += performance.now() - started }
82
+ signal?.throwIfAborted()
83
+ const status = page.relays?.find(item => item.relay === relay)
84
+ if (page.errors?.length || !status || !['eose', 'satisfied'].includes(status.status)) {
85
+ await process(page.result)
86
+ errors.push(...(page.errors ?? []))
87
+ outcome = status ?? { relay, status: 'error' }
88
+ break
89
+ }
90
+ if (status.status === 'satisfied' || page.result.length >= interval.limit) {
91
+ if (interval.since < interval.until) {
92
+ const middle = Math.floor(interval.since + (interval.until - interval.since) / 2)
93
+ intervals.push({ since: middle + 1, until: interval.until, limit: PAGE_SIZE }, { since: interval.since, until: middle, limit: PAGE_SIZE })
94
+ } else if (interval.limit < MAX_PAGE_SIZE) {
95
+ intervals.push({ ...interval, limit: interval.limit * 2 })
96
+ } else {
97
+ await process(page.result)
98
+ const error = pageLimitError()
99
+ errors.push({ relay, reason: error })
100
+ outcome = { relay, status: 'error', error }
101
+ break
102
+ }
103
+ } else {
104
+ await process(page.result)
105
+ }
106
+ signal?.throwIfAborted()
107
+ } finally {
108
+ releasePage?.()
109
+ releaseRelay?.()
110
+ }
111
+ }
112
+ report.push(outcome)
113
+ }
114
+ signal?.throwIfAborted()
115
+ if (errors.length || report.some(item => item.status !== 'eose')) {
116
+ const error = incompleteFetchError({ errors, report, receivedEventCount, elapsedMs: Math.round(elapsedMs), request: { relays: urls, channelPubkeys: filter.authors ?? [], receiverPubkey, since, until, limit: PAGE_SIZE, timeoutMs: TIMEOUT_MS } })
117
+ error.operation = 'private-channel.fetchHistory'
118
+ throw error
119
+ }
120
+ return { oldestCreatedAt, receivedEventCount, relays: report }
121
+ }
@@ -7,6 +7,7 @@ import { getIykcProofs } from '../content-key/index.js'
7
7
  import { ValidationError } from '../error/index.js'
8
8
  import { normalizeRumor, deliveryInfo } from './helpers/rumor.js'
9
9
  import { incompleteFetchError } from './helpers/fetch-error.js'
10
+ import { readHistory } from './helpers/history.js'
10
11
  import { subscriptionError } from './helpers/subscription-error.js'
11
12
  import * as nip44v3 from '../nip44-v3/index.js'
12
13
  import { relayPool } from '../relay/index.js'
@@ -37,7 +38,7 @@ const encoder = new TextEncoder()
37
38
  const decoder = new TextDecoder()
38
39
  const NIP44_V3_SCOPE = ''
39
40
 
40
- const sendToRelays = (...args) => relayPool.sendEvent(...args)
41
+ const sendToRelays = (event, relays) => relayPool.sendEvent(event, relays)
41
42
  const getEvents = (...args) => relayPool.getEvents(...args)
42
43
  const getLiveEventsGenerator = (...args) => relayPool.getLiveEventsGenerator(...args)
43
44
  const getEventsFeedGenerator = (...args) => relayPool.getEventsFeedGenerator(...args)
@@ -540,7 +541,7 @@ export async function publish ({ senderSigner, imkcSigner, privateChannelSigner
540
541
  try {
541
542
  for (const group of groups) {
542
543
  for await (const wrappedEvent of wrapPreparedEvents({ privateChannelSigner, receivers: group.receivers, receiverTag, fileChunkIndex, onPreparedSeed, deletionPubkey: normalizedDeletionPubkey, expirationSeconds, context })) {
543
- results.push(await _publish(wrappedEvent, withRecoveryRelays(group.relays, recoveryRelays)))
544
+ results.push(await _publish(wrappedEvent, withRecoveryRelays(group.relays, recoveryRelays), { receiverPubkeys: receiverPubkeys(group.receivers), primaryRelays: group.relays }))
544
545
  }
545
546
  }
546
547
  } finally {
@@ -550,7 +551,7 @@ export async function publish ({ senderSigner, imkcSigner, privateChannelSigner
550
551
  }
551
552
 
552
553
  for await (const wrappedEvent of wrapEvents({ senderSigner, imkcSigner, privateChannelSigner, privateChannelReaderPubkey, receivers, receiverTag, fileChunkIndex, onPreparedSeed, deletionPubkey: normalizedDeletionPubkey, event, expirationSeconds, temporaryStorageArea, _getIykcProofs })) {
553
- results.push(await _publish(wrappedEvent, withRecoveryRelays(relays, recoveryRelays)))
554
+ results.push(await _publish(wrappedEvent, withRecoveryRelays(relays, recoveryRelays), { receiverPubkeys: receiverPubkeys(receivers), primaryRelays: relays }))
554
555
  }
555
556
  return results
556
557
  }
@@ -955,6 +956,17 @@ export async function fetch ({ signal, receiverSigner, iykcSigner, privateChanne
955
956
  }
956
957
  }
957
958
 
959
+ // Complete interval recovery with bounded pages and one fragment processor.
960
+ // Existing fetch() retains its one-query/event-array contract.
961
+ export async function fetchHistory ({ receiverSigner, privateChannelSigner = receiverSigner, _getEvents = getEvents, _acquirePage, ...options }) {
962
+ const authors = privateChannelPubkeyList(options)
963
+ const filter = { kinds: [PRIVATE_BROADCAST_KIND], ...(authors.length ? { authors } : {}), since: options.since, until: options.until }
964
+ const processOuterEvent = createProcessor({ ...options, receiverSigner, privateChannelSigner, onError: error => { options.onError?.(error); throw error } })
965
+ try {
966
+ return await readHistory({ filter, relays: options.relays, receiverPubkey: options.receiverPubkey, signal: options.signal, getEvents: _getEvents, processEvent: processOuterEvent, acquirePage: _acquirePage })
967
+ } finally { processOuterEvent.close() }
968
+ }
969
+
958
970
  export function subscribe ({ receiverSigner, iykcSigner, privateChannelSigner = receiverSigner, privateChannelSignersByPubkey, privateChannelReaderSigner = privateChannelSigner, privateChannelReaderSignersByPubkey, privateChannelReaderPubkey, privateChannelReaderPubkeysByPubkey, privateChannelPubkey, privateChannelPubkeys, receiverPubkey, relays, onChunk, onEvent, onNymEvent, onSeedEvent, onContentKeyUsage, onError, since = nowSeconds() - 5, limit, liveOnly = false, mode = 'leecher', modeByPubkey, receivedChunkScope = '', receivedChunkTtlMs = DEFAULT_RECEIVED_CHUNK_TTL_MS, receivedChunkTtlMsByPubkey, receivedChunkMaxBytes = DEFAULT_RECEIVED_CHUNK_MAX_BYTES, receivedChunkIndexedDB = globalThis.indexedDB, ignoredGroupTtlMs = DEFAULT_IGNORED_GROUP_TTL_MS, ignoredGroupMaxEntries = DEFAULT_IGNORED_GROUP_MAX_ENTRIES, _liveEventsGenerator = getLiveEventsGenerator, _eventsFeedGenerator = getEventsFeedGenerator }) {
959
971
  if (!relays?.length) throw new ValidationError('NO_RELAYS')
960
972
  if (receiverSigner && !receiverSigner?.nip44DecryptDoubleDH && !receiverSigner?.nip44v3Decrypt) throw new ValidationError('RECEIVER_SIGNER_NIP44V3_DECRYPT_UNSUPPORTED')
@@ -1,7 +1,8 @@
1
1
  import { getEventHash } from '../../event/index.js'
2
2
 
3
3
  // A report describes one outer event, not one relay. Each outer event needs
4
- // at least one relay acknowledgement. Keep the successful fast path immediate.
4
+ // accepted publication covering its routed recipients. Keep the successful fast
5
+ // path immediate; a partially covered multi-recipient report remains a failure.
5
6
  export async function assertMessagePublished (result, event) {
6
7
  const reports = result?.delivery?.reports
7
8
  const present = Array.isArray(reports) && reports.length > 0
@@ -1,4 +1,5 @@
1
1
  import { normalizeRelayUrl } from '../../url/index.js'
2
+ import { ValidationError } from '../../error/index.js'
2
3
 
3
4
  const REJECTION_PREFIXES = new Set(['blocked', 'restricted', 'auth-required', 'pow', 'rate-limited', 'error'])
4
5
  const TRANSPORT_CATEGORIES = new Set(['connection', 'transport', 'timeout'])
@@ -14,17 +15,60 @@ export function isReplaceableRelayFailure (error) {
14
15
  return REJECTION_PREFIXES.has(/^([a-z-]+):/.exec(error?.message || '')?.[1])
15
16
  }
16
17
 
17
- export function createSendRelayRouting ({ peer, relaysByPubkey, exclusions, pickRelays, recoveryRelays, publish, publishNymEvent, sendEvent, isOnline, isCurrent, signal, now = Date.now }) {
18
- // One finite snapshot, including the existing default pair if NIP-65 is
19
- // unavailable. Never invent additional fallback relays.
20
- const candidates = [...pickRelays([peer], relaysByPubkey, { relayType: 'read', maxPerPubkey: Infinity }).keys()]
18
+ export function normalizeFallbackRelays (value = []) {
19
+ if (!Array.isArray(value)) throw new ValidationError('INVALID_FALLBACK_RELAYS')
20
+ return [...new Set(Array.from(value, normalizeRelayUrl))]
21
+ }
22
+
23
+ function routeEntries (routes) {
24
+ return routes instanceof Map ? [...routes] : Object.entries(routes || {})
25
+ }
26
+
27
+ function routeGroups (routes) {
28
+ const groups = new Map()
29
+ for (const [relay, receivers] of routes) {
30
+ const key = [...receivers].sort().join(',')
31
+ if (!groups.has(key)) groups.set(key, { receivers, relays: [] })
32
+ groups.get(key).relays.push(relay)
33
+ }
34
+ return [...groups.values()]
35
+ }
36
+
37
+ export function createSendRelayRouting ({ peer, peers = [peer], relaysByPubkey, primaryRelays, primaryRelayToReceivers, fallbackRelays = [], exclusions, pickRelays, recoveryRelays, publish, publishNymEvent, sendEvent, isOnline, isCurrent, signal, now = Date.now }) {
38
+ // Keep discovery's defaults when NIP-65 is unavailable. Explicit maps/lists
39
+ // retain their fanout; configured fallbacks are a separate last-resort stage.
40
+ const explicit = Boolean(primaryRelays || primaryRelayToReceivers)
41
+ const primaryRoutes = primaryRelayToReceivers || (primaryRelays
42
+ ? new Map(primaryRelays.map(relay => [relay, peers]))
43
+ : pickRelays(peers, relaysByPubkey, { relayType: 'read', maxPerPubkey: Infinity }))
44
+ const primaryByPeer = Object.fromEntries(peers.map(pubkey => [pubkey, { read: [] }]))
45
+ for (const [relay, receivers] of routeEntries(primaryRoutes)) {
46
+ for (const pubkey of Array.isArray(receivers) ? receivers : [receivers]) primaryByPeer[pubkey]?.read.push(normalizeRelayUrl(relay))
47
+ }
48
+ const candidates = [...new Set([...Object.values(primaryByPeer).flatMap(value => value.read), ...fallbackRelays])]
21
49
  const candidateSet = new Set(candidates)
22
50
  for (const [relay, expires] of exclusions) if (expires <= now() || !candidateSet.has(relay)) exclusions.delete(relay)
23
- // New attempts may recheck an exhausted list. Each in-flight outer event has
24
- // its own tried set, preventing loops during concurrent sends.
25
- if (candidates.length && candidates.every(relay => exclusions.has(relay))) exclusions.clear()
26
- const select = excluded => pickRelays([peer], relaysByPubkey, { relayType: 'read', excludeRelaysByPubkey: new Map([[peer, [...excluded]]]) })
27
- const relayToReceivers = select(exclusions.keys())
51
+ // A fresh attempt can recheck an exhausted list; each outer event still owns
52
+ // a tried set so concurrent sends/fragments cannot create a retry loop.
53
+ if (peers.some(pubkey => {
54
+ const available = [...primaryByPeer[pubkey].read, ...fallbackRelays]
55
+ return available.length && available.every(relay => exclusions.has(relay))
56
+ })) exclusions.clear()
57
+ const select = (receivers, excluded, staged = false) => {
58
+ const excludedRelays = [...excluded]
59
+ const excludeRelaysByPubkey = new Map(receivers.map(pubkey => [pubkey, excludedRelays]))
60
+ const options = { relayType: 'read', excludeRelaysByPubkey, emptyRelaysFallback: [] }
61
+ const routes = pickRelays(receivers, primaryByPeer, { ...options, ...(explicit ? { maxPerPubkey: Infinity } : {}) })
62
+ // For one ciphertext, exhaust the remaining members' primary routes before
63
+ // using a common fallback. Initial wrapping must nevertheless cover everyone.
64
+ if (staged && routes.size) return routes
65
+ const covered = new Set([...routes.values()].flat())
66
+ const missing = receivers.filter(pubkey => !covered.has(pubkey))
67
+ const fallback = pickRelays(missing, Object.fromEntries(missing.map(pubkey => [pubkey, { read: fallbackRelays }])), options)
68
+ for (const [relay, pubkeys] of fallback) routes.set(relay, [...new Set([...(routes.get(relay) || []), ...pubkeys])])
69
+ return routes
70
+ }
71
+ const relayToReceivers = primaryRelayToReceivers || select(peers, exclusions.keys())
28
72
  const current = () => !signal?.aborted && isCurrent()
29
73
  const rejected = report => (report?.errors || []).filter(item => candidateSet.has(normalized(item.relay)) && isReplaceableRelayFailure(item.reason))
30
74
  const remember = errors => { for (const item of errors) exclusions.set(normalized(item.relay), now() + EXCLUSION_MS) }
@@ -33,58 +77,68 @@ export function createSendRelayRouting ({ peer, relaysByPubkey, exclusions, pick
33
77
  try { return await isOnline({ signal }) && current() } catch { return false }
34
78
  }
35
79
 
36
- async function send (event, initialRelays) {
80
+ async function send (event, initialRelays, context = {}) {
81
+ // This context describes the already encrypted recipient subset. It stays
82
+ // local: no recipient tags or routing metadata are added to the outer event.
83
+ const receivers = context.receiverPubkeys || peers
84
+ const pending = new Set(receivers)
37
85
  const mirrors = initialRelays.filter(relay => !candidateSet.has(normalized(relay)))
38
- let relays = [...new Set([...select(exclusions.keys()).keys(), ...mirrors])]
86
+ const initialPrimary = context.primaryRelays || (primaryRelays || peers.length === 1 ? routeEntries(relayToReceivers).map(([relay]) => relay) : [])
87
+ const first = [...new Set(initialPrimary.map(normalizeRelayUrl))].filter(relay => !exclusions.has(relay))
88
+ let batch = first.length ? { receivers, relays: first } : null
39
89
  const tried = new Set()
40
90
  const errors = []
41
91
  const summaries = []
42
- const finish = result => {
43
- if (summaries.length === 1) return result
44
- return {
45
- ...result,
92
+ const next = () => routeGroups(select([...pending], new Set([...exclusions.keys(), ...tried]), true))[0]
93
+ const finish = result => ({
94
+ ...result,
95
+ success: summaries.length > 0 && pending.size === 0,
96
+ total: tried.size,
97
+ promise: Promise.all(summaries).then(reports => ({
46
98
  total: tried.size,
47
- promise: Promise.all(summaries).then(reports => ({
48
- total: tried.size,
49
- success: reports.some(report => report.success),
50
- fulfilled: reports.reduce((total, report) => total + (report.fulfilled || 0), 0),
51
- succeededRelays: [...new Set(reports.flatMap(report => report.succeededRelays || []))],
52
- errors: reports.flatMap(report => report.errors || [])
53
- }))
54
- }
55
- }
56
- while (true) {
99
+ success: summaries.length > 0 && pending.size === 0,
100
+ fulfilled: reports.reduce((total, report) => total + (report.fulfilled || 0), 0),
101
+ succeededRelays: [...new Set(reports.flatMap(report => report.succeededRelays || []))],
102
+ errors: reports.flatMap(report => report.errors || [])
103
+ }))
104
+ })
105
+ let result = { success: false }
106
+ while ((batch ||= next())) {
57
107
  signal?.throwIfAborted()
108
+ const relays = [...new Set([...batch.relays, ...(summaries.length ? [] : mirrors)])]
58
109
  for (const relay of relays) tried.add(normalized(relay))
59
- // Republish the exact signed outer event, preserving encryption,
60
- // fragments, timestamps and deletion capabilities.
61
- const result = await sendEvent(event, relays)
110
+ // Every relay in this batch serves the same pending recipient subset.
111
+ // One ACK covers that subset, never unrelated members of the ciphertext.
112
+ result = await sendEvent(event, relays)
62
113
  const settled = Promise.resolve(result.promise)
63
114
  summaries.push(settled)
64
115
  if (result.success) {
116
+ for (const pubkey of batch.receivers) pending.delete(pubkey)
65
117
  // Keep the first ACK immediate; learn about redundant failures later.
66
118
  settled.then(async report => {
67
119
  const failed = rejected(report)
68
120
  if (failed.length && await online()) remember(failed)
69
121
  }).catch(() => {})
70
- return finish(result)
122
+ if (!pending.size) return finish(result)
123
+ } else {
124
+ const report = await settled
125
+ const failed = rejected(report)
126
+ errors.push(...(report?.errors || []))
127
+ if (!failed.length || errors.some(item => !isReplaceableRelayFailure(item.reason))) return finish(result)
128
+ if (!next()) { exclusions.clear(); return finish(result) }
129
+ if (!await online()) return { ...finish(result), retryWhenAvailable: !signal?.aborted, retryWhenOnline: current() }
130
+ remember(failed)
71
131
  }
72
- const report = await settled
73
- const failed = rejected(report)
74
- errors.push(...(report?.errors || []))
75
- if (!failed.length || errors.some(item => !isReplaceableRelayFailure(item.reason))) return finish(result)
76
- const next = [...select(new Set([...exclusions.keys(), ...tried])).keys()]
77
- if (!next.length) { exclusions.clear(); return finish(result) }
78
- if (!await online()) return { ...finish(result), retryWhenAvailable: !signal?.aborted, retryWhenOnline: current() }
79
- remember(failed)
80
- relays = next
132
+ if (!current()) return { ...finish(result), retryWhenAvailable: !signal?.aborted, retryWhenOnline: false }
133
+ batch = null
81
134
  }
135
+ return finish(result)
82
136
  }
83
137
 
84
138
  return {
85
139
  relayToReceivers,
86
- // A recovery mirror must not reintroduce the recipient's refused relays.
87
- recoveryRelays: recoveryRelays.filter(relay => !candidateSet.has(relay) || !exclusions.has(relay)),
140
+ // A recovery mirror must not reintroduce refused relays or promote a fallback.
141
+ recoveryRelays: recoveryRelays.filter(relay => !candidateSet.has(normalized(relay)) || !exclusions.has(normalized(relay))),
88
142
  _publish: options => (options.nymSigner ? publishNymEvent : publish)({ ...options, _publish: send })
89
143
  }
90
144
  }
@@ -42,9 +42,11 @@
42
42
  // - Seeder replies stream compact routers with createMissingMessageReplyPacker({ messenger, question }).update(seed), then finalize(optionalLastSeed).
43
43
  // - For other event-list replies, use createEventReplyPacker({ messenger, question, code }).update(event).
44
44
 
45
+ import { createAbortableSemaphore } from '../helpers/abortable-semaphore.js'
45
46
  import * as privateMessage from '../private-message/index.js'
46
47
  import { isOnline } from '../network/index.js'
47
- import { createSendRelayRouting } from './helpers/send-routing.js'
48
+ import { createSendRelayRouting, normalizeFallbackRelays } from './helpers/send-routing.js'
49
+ import { normalizeRelayUrl } from '../url/index.js'
48
50
  import { deliveryInfo } from '../private-channel/helpers/rumor.js'
49
51
  import { bytesToBase64 } from '../base64/index.js'
50
52
  import { ValidationError } from '../error/index.js'
@@ -193,6 +195,7 @@ export class PrivateMessenger {
193
195
  }
194
196
 
195
197
  constructor ({
198
+ fallbackRelays = [],
196
199
  offlineRecoverySeconds = DEFAULT_OFFLINE_RECOVERY_SECONDS,
197
200
  staleChannelSeconds = DEFAULT_STALE_CHANNEL_SECONDS,
198
201
  identityStorageRetentionSeconds = DEFAULT_IDENTITY_STORAGE_RETENTION_SECONDS,
@@ -226,6 +229,7 @@ export class PrivateMessenger {
226
229
  _storageClearInterval = globalThis.clearInterval.bind(globalThis),
227
230
  _BroadcastChannel = _indexedDB === globalThis.indexedDB ? globalThis.BroadcastChannel : undefined
228
231
  } = {}) {
232
+ this.fallbackRelays = normalizeFallbackRelays(fallbackRelays)
229
233
  this.offlineRecoverySeconds = normalizeOfflineRecoverySeconds(offlineRecoverySeconds)
230
234
  this.staleChannelSeconds = normalizeStaleChannelSeconds(staleChannelSeconds)
231
235
  this.identityStorageRetentionSeconds = normalizeIdentityStorageRetentionSeconds(identityStorageRetentionSeconds)
@@ -282,6 +286,7 @@ export class PrivateMessenger {
282
286
  this.deliveryReads = new Set()
283
287
  this.deliveryWaiters = new Set()
284
288
  this.recoveries = new Map()
289
+ this.recoveryAdmission = createAbortableSemaphore(2)
285
290
  this.recoveryControllers = new Set()
286
291
  this.resumeWork = null
287
292
  this.capacityTimer = null
@@ -756,42 +761,38 @@ export class PrivateMessenger {
756
761
  }
757
762
 
758
763
  async resolveWatchRelays (channel) {
759
- if (!channel.usesNip65WatchRelays && channel.relays.length) return channel.relays
760
- return this.readRelaysForPubkey(this.userPubkey)
764
+ const primary = !channel.usesNip65WatchRelays && channel.relays.length ? channel.relays : await this.readRelaysForPubkey(this.userPubkey)
765
+ return uniq([...primary, ...this.fallbackRelays].map(normalizeRelayUrl))
761
766
  }
762
767
 
763
768
  async resolveSendRouting ({ channel, receiverPubkeys, relays, relayToReceivers, signal }) {
764
769
  const recoveryRelays = await this.recoveryMirrorRelays(channel.pubkey)
765
- if (relayToReceivers) return { relayToReceivers, recoveryRelays }
766
- if (relays?.length) return { relays: uniq(relays), recoveryRelays }
767
- if (channel.sendRelays.length) return { relays: channel.sendRelays, recoveryRelays }
768
- if (channel.relays.length) return { relays: channel.relays, recoveryRelays }
769
- const recipients = uniq(receiverPubkeys)
770
- if (recipients.length === 1) {
771
- const peer = recipients[0]
772
- const relaysByPubkey = await this._getRelaysByPubkey(recipients)
773
- const key = `${channel.pubkey}:${peer}`
774
- if (!this.sendRelayExclusions.has(key)) {
775
- if (this.sendRelayExclusions.size >= 256) this.sendRelayExclusions.delete(this.sendRelayExclusions.keys().next().value)
776
- this.sendRelayExclusions.set(key, { channelPubkey: channel.pubkey, exclusions: new Map() })
777
- }
778
- const routing = createSendRelayRouting({
779
- peer, relaysByPubkey, recoveryRelays,
780
- signal: signal ? AbortSignal.any([this.sendRoutingLifetime.signal, signal]) : this.sendRoutingLifetime.signal,
781
- exclusions: this.sendRelayExclusions.get(key).exclusions,
782
- pickRelays: this._pickRelaysForPubkeys,
783
- publish: this._privateChannel.publish || privateChannel.publish,
784
- publishNymEvent: this._privateChannel.publishNymEvent || privateChannel.publishNymEvent,
785
- sendEvent: (...args) => relayPool.sendEvent(...args),
786
- isOnline: this._isOnline,
787
- isCurrent: () => !this.closePromise && !this.pauseReasons.size && this.channels.has(channel.pubkey)
788
- })
789
- if (!relayMapRelays(routing.relayToReceivers).length) throw new ValidationError('NO_RELAYS')
790
- return routing
770
+ if (relayToReceivers && !this.fallbackRelays.length) return { relayToReceivers, recoveryRelays }
771
+ const fixed = relayToReceivers ? null : relays?.length ? uniq(relays) : channel.sendRelays.length ? channel.sendRelays : channel.relays.length ? channel.relays : null
772
+ if (fixed && !this.fallbackRelays.length) return { relays: fixed, recoveryRelays }
773
+ const recipients = uniq(receiverPubkeys?.length ? receiverPubkeys : relayMapReceivers(relayToReceivers))
774
+ const relaysByPubkey = fixed || relayToReceivers ? undefined : await this._getRelaysByPubkey(recipients)
775
+ const key = `${channel.pubkey}:${[...recipients].sort().join(',')}`
776
+ if (!this.sendRelayExclusions.has(key)) {
777
+ if (this.sendRelayExclusions.size >= 256) this.sendRelayExclusions.delete(this.sendRelayExclusions.keys().next().value)
778
+ this.sendRelayExclusions.set(key, { channelPubkey: channel.pubkey, exclusions: new Map() })
791
779
  }
792
- const derived = await this.readRelayToReceivers(recipients)
793
- if (!relayMapRelays(derived).length) throw new ValidationError('NO_RELAYS')
794
- return { relayToReceivers: derived, recoveryRelays }
780
+ const routing = createSendRelayRouting({
781
+ peers: fixed && !recipients.length ? [this.userPubkey] : recipients, relaysByPubkey, recoveryRelays, primaryRelays: fixed?.map(normalizeRelayUrl), primaryRelayToReceivers: relayToReceivers, fallbackRelays: this.fallbackRelays,
782
+ signal: signal ? AbortSignal.any([this.sendRoutingLifetime.signal, signal]) : this.sendRoutingLifetime.signal,
783
+ exclusions: this.sendRelayExclusions.get(key).exclusions,
784
+ pickRelays: this._pickRelaysForPubkeys,
785
+ publish: this._privateChannel.publish || privateChannel.publish,
786
+ publishNymEvent: this._privateChannel.publishNymEvent || privateChannel.publishNymEvent,
787
+ sendEvent: (...args) => relayPool.sendEvent(...args),
788
+ isOnline: this._isOnline,
789
+ isCurrent: () => !this.closePromise && !this.pauseReasons.size && this.channels.has(channel.pubkey)
790
+ })
791
+ if (!relayMapRelays(routing.relayToReceivers).length) throw new ValidationError('NO_RELAYS')
792
+ // Fixed lists address the original receiver set as a whole, preserving the
793
+ // existing encryption grouping instead of inventing per-recipient routes.
794
+ if (fixed) return { relays: fixed, recoveryRelays: routing.recoveryRelays, _publish: routing._publish }
795
+ return routing
795
796
  }
796
797
 
797
798
  readState () {
@@ -1951,12 +1952,11 @@ export class PrivateMessenger {
1951
1952
  }
1952
1953
 
1953
1954
  async askSeedersForRelayLeftEdge (channelPubkey, range, fetchedEvents) {
1954
- const { asks } = await this.#askSeedersForRelayLeftEdgeAttempt(channelPubkey, range, fetchedEvents)
1955
+ const { asks } = await this.#askSeedersForRelayLeftEdgeAttempt(channelPubkey, range, oldestCreatedAt(fetchedEvents))
1955
1956
  return asks
1956
1957
  }
1957
1958
 
1958
- async #askSeedersForRelayLeftEdgeAttempt (channelPubkey, range, fetchedEvents) {
1959
- const oldest = oldestCreatedAt(fetchedEvents)
1959
+ async #askSeedersForRelayLeftEdgeAttempt (channelPubkey, range, oldest) {
1960
1960
  const until = oldest == null ? range.end : Math.min(range.end, oldest)
1961
1961
  if (until < range.start) return { asks: [], failures: [] }
1962
1962
  return this.#askSeedersForMissingRangeAttempt(channelPubkey, range.start, until)
@@ -2099,7 +2099,8 @@ export class PrivateMessenger {
2099
2099
  if (this.pauseReasons.size || this.closePromise || !this.desiredChannels.has(pubkey)) throw new Error('PRIVATE_MESSENGER_PAUSED')
2100
2100
  const fetchRelays = await this.resolveWatchRelays(channel)
2101
2101
  controller.signal.throwIfAborted()
2102
- const fetchedEvents = await this._privateChannel.fetch({
2102
+ const history = await this._privateChannel.fetchHistory({
2103
+ _acquirePage: signal => this.recoveryAdmission.acquire(signal),
2103
2104
  signal: controller.signal,
2104
2105
  receivedChunkScope: this.storageLeaseId,
2105
2106
  receiverSigner: this.userSigner,
@@ -2121,9 +2122,9 @@ export class PrivateMessenger {
2121
2122
  onSeedEvent: seed => this.receive(pubkey, () => this.enqueueSeed(pubkey, seed), seed),
2122
2123
  onContentKeyUsage: usage => this.handleContentKeyUsage(pubkey, usage),
2123
2124
  onError: err => { throw err }
2124
- }) || []
2125
+ })
2125
2126
  controller.signal.throwIfAborted()
2126
- const attempt = await this.#askSeedersForRelayLeftEdgeAttempt(pubkey, range, fetchedEvents)
2127
+ const attempt = await this.#askSeedersForRelayLeftEdgeAttempt(pubkey, range, history.oldestCreatedAt)
2127
2128
  const lifecycleChanged = this.closePromise || !this.channels.has(pubkey) || !this.stopByChannel.has(pubkey) ||
2128
2129
  (this.watchRevisionByChannel.get(pubkey) || 0) !== watchRevision
2129
2130
  if (lifecycleChanged || attempt.failures.length) remaining.push(range)
@@ -2330,6 +2331,12 @@ function oldestCreatedAt (events) {
2330
2331
  return oldest
2331
2332
  }
2332
2333
 
2334
+ function relayMapReceivers (relayToReceivers) {
2335
+ if (!relayToReceivers) return []
2336
+ const values = relayToReceivers instanceof Map ? [...relayToReceivers.values()] : Object.values(relayToReceivers)
2337
+ return uniq(values.flat())
2338
+ }
2339
+
2333
2340
  function relayMapRelays (relayToReceivers) {
2334
2341
  if (!relayToReceivers) return []
2335
2342
  const entries = relayToReceivers instanceof Map ? relayToReceivers.entries() : Object.entries(relayToReceivers)
@@ -48,21 +48,53 @@ This is a DM profile; group membership/key derivation is not implemented.
48
48
 
49
49
  ## Relay fallback
50
50
 
51
- Automatic single-recipient routing uses the recipient's NIP-65 read relays, at
52
- most two per attempt. On `blocked`, `restricted`, `auth-required`, `pow`,
53
- `rate-limited`, or `error` rejections, or connection/transport/timeout failures,
54
- it can try remaining read relays after `isOnline` confirms connectivity. Invalid
55
- messages, local signer/authentication failures and unknown error text retain their
56
- original diagnosis. Explicit per-call `relays`/`relayToReceivers`, channel
57
- `sendRelays`/`relays`, and multi-recipient sends do not enable automatic fallback.
51
+ Both `createPrivateMessenger` and `createPrivateMessageSession` accept an optional
52
+ `fallbackRelays` array (default `[]`) at construction. URLs are normalized,
53
+ deduplicated and copied; invalid values throw `ValidationError`. For example:
54
+
55
+ ```js
56
+ const session = createPrivateMessageSession({
57
+ owner, signer, eventStore,
58
+ fallbackRelays: ['wss://relay.44billion.net']
59
+ })
60
+ ```
61
+
62
+ Automatic sends try each recipient's NIP-65 read relays first, at most two per
63
+ recipient per attempt, then the configured fallbacks in pairs. Explicit
64
+ per-call `relays`, channel `sendRelays`/`relays` and global `relays` remain primary
65
+ and also support configured fallbacks, including fixed multi-recipient sends.
66
+ Explicit lists retain their requested initial fanout; no recipient relay lookup
67
+ is added. A relay present in both lists is attempted only once per outer event.
68
+ Automatic multi-recipient sends and explicit `relayToReceivers` maps also support
69
+ fallbacks. Explicit maps retain their precedence over fixed lists, initial fanout
70
+ and encrypted recipient subsets, without consulting recipient NIP-65 lists.
71
+ The session coordinator remains a DM API; multi-recipient channels use
72
+ `createPrivateMessenger` directly.
73
+
74
+ On `blocked`, `restricted`, `auth-required`, `pow`, `rate-limited`, or `error`
75
+ rejections, or connection/transport/timeout failures, replacement requires
76
+ `isOnline` to confirm connectivity. Invalid messages, local signer/authentication
77
+ failures and unknown error text retain their original diagnosis.
78
+
79
+ Receive routing always adds the configured fallback relays to the effective
80
+ primary list (explicit `relays` or the owner's NIP-65 read relays). Listen there
81
+ from the start, even when primary relays are healthy: a sender may have used a
82
+ fallback after publication was refused elsewhere. Live watches, historical
83
+ recovery and file-channel reads share this policy. NIP-65 refreshes retain the
84
+ fallbacks; outgoing exclusions never remove them from receive subscriptions.
85
+ No public recipient tags or changes to a user's NIP-65 event are introduced.
58
86
 
59
87
  Each signed outer event is reused verbatim, including router/carrier fragments
60
88
  and deletion capabilities. Native errors from exhausted attempts remain in its
61
- publication report. One successful relay acknowledgement completes that outer
62
- event immediately; other acknowledgements may update future routing preferences
63
- without delaying the send. Preferences are channel/recipient-scoped, in-memory,
89
+ publication report. A shared relay's first ACK covers all members assigned to
90
+ that publication batch. If replacements use different relays for different
91
+ members, every pending member must be covered before the outer event succeeds;
92
+ a partial ACK cannot hide another member's failure. Remaining primary routes are
93
+ exhausted before configured fallbacks. Redundant acknowledgements may update
94
+ future routing preferences without delaying a successfully covered send. Preferences are channel/recipient-scoped, in-memory,
64
95
  expire after five minutes, and never alter receive subscriptions. A fresh attempt
65
- can recheck an exhausted list. No additional public fallback relays are invented.
96
+ can recheck an exhausted list. Additional fallback relays come only from the
97
+ caller configuration; absent NIP-65 lists retain the existing discovery defaults.
66
98
 
67
99
  When offline/interrupted with alternatives remaining, reports carry
68
100
  `retryWhenAvailable: true`. The outbox remains pending and emits no `onSendError`.
@@ -8,6 +8,7 @@ import { decodeFileMetadata } from '../../nip94/index.js'
8
8
  import { createChatOutbox } from './helpers/work-storage.js'
9
9
  import { messengerSigner } from './helpers/signer.js'
10
10
  import { assertMessagePublished } from '../helpers/publication.js'
11
+ import { normalizeFallbackRelays } from '../helpers/send-routing.js'
11
12
 
12
13
  export function wireEvent (value, owner) {
13
14
  const event = { kind: value.kind, created_at: value.created_at, tags: structuredClone(value.tags), content: value.content, pubkey: value.pubkey || owner }
@@ -17,7 +18,8 @@ export function wireEvent (value, owner) {
17
18
  // Relay rejection text must not be interpreted as a local signer denial.
18
19
  const retryable = error => error?.code === 'MESSAGE_NOT_PUBLISHED' || !/DENIED|PERMISSION|REVOKED|READ_ONLY|INVALID|BLOCKED|EXPIRED|NOT_IN_PERSONA/i.test(`${error?.code || ''} ${error?.message || ''}`)
19
20
 
20
- export function createPrivateMessageSession ({ owner, signer, eventStore, messageStorage, chunkStorage, recoveryStorage, mode = 'seeder', seedersForPeer = peer => [peer], allowedKinds = [5, 9, 1063, 34601], onMedia = () => {}, onOutbox = () => {}, onError = () => {}, onSendError = () => {}, Messenger = createPrivateMessenger, openOutbox = createChatOutbox, FileTransfer = createPrivateFileTransfer, _onOnline = onOnline, openDownloads = options => createChatOutbox({ ...options, namespace: 'downloads' }) }) {
21
+ export function createPrivateMessageSession ({ owner, signer, eventStore, messageStorage, chunkStorage, recoveryStorage, fallbackRelays = [], mode = 'seeder', seedersForPeer = peer => [peer], allowedKinds = [5, 9, 1063, 34601], onMedia = () => {}, onOutbox = () => {}, onError = () => {}, onSendError = () => {}, Messenger = createPrivateMessenger, openOutbox = createChatOutbox, FileTransfer = createPrivateFileTransfer, _onOnline = onOnline, openDownloads = options => createChatOutbox({ ...options, namespace: 'downloads' }) }) {
22
+ fallbackRelays = normalizeFallbackRelays(fallbackRelays)
21
23
  const userSigner = messengerSigner(signer)
22
24
  messageStorage ||= createEventStoreMessageStorage({ eventStore })
23
25
  chunkStorage ||= eventStore ? createEventStoreChunkStorage({ eventStore }) : undefined
@@ -74,7 +76,7 @@ export function createPrivateMessageSession ({ owner, signer, eventStore, messag
74
76
  if (closed || !available || version !== lifecycle) return
75
77
  values.push(channel)
76
78
  }
77
- if (!messenger) messenger = await Messenger({ seedStorage: recoveryStorage?.seeds, userSigner, channels: [], useContentKeys: false, onMessageQueued: () => drain(), onError })
79
+ if (!messenger) messenger = await Messenger({ fallbackRelays, seedStorage: recoveryStorage?.seeds, userSigner, channels: [], useContentKeys: false, onMessageQueued: () => drain(), onError })
78
80
  if (closed || !available || version !== lifecycle) { await messenger.pause('signer'); return }
79
81
  await messenger.update({ channels: values })
80
82
  await messenger.resume('signer')