libp2r2p 0.11.12 → 0.11.14

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -877,9 +877,42 @@ the first pass routes through up to `maxPerPubkey` shared relays, and missing
877
877
  authors are retried on every remaining relay plus `fallbackRelays` (default:
878
878
  the first three `freeRelays`), excluding what was already queried. It accepts
879
879
  `relaysByPubkey` to reuse a previous discovery (only missing pubkeys are then
880
- discovered) and returns `{ events, byPubkey, relaysByPubkey }` so the merged
880
+ discovered) and returns `{ events, byPubkey, relaysByPubkey, requests }` so the merged
881
881
  relay map can be passed back on later calls.
882
882
 
883
+
884
+ Read diagnostics preserve native failures rather than treating an empty result
885
+ as proof that every relay completed successfully. `requests` contains records
886
+ `{ phase, authors, relays, errors?, error? }`: phase is `discovery`, `primary` or
887
+ `fallback`; `relays` uses the pool's `{ relay, status, error? }` outcomes. An
888
+ operation-wide exception is retained as `error`. Events remain in the existing
889
+ result fields, never in diagnostic records. `onQueryResult(record)` receives each
890
+ record as the underlying query settles, including before a later cancellation.
891
+
892
+ Both discovery and latest-event queries accept `signal`. Cancelling one consumer
893
+ of shared NIP-65 discovery releases only its interest; the underlying query is
894
+ aborted when its last consumer leaves. Discovery's optional `onQueryResult`
895
+ receives the same native report for every joining caller. Incomplete/failed
896
+ lookups can use fallback relays for that call but never negatively cache a
897
+ missing relay list for forty minutes. Actual lists and completely empty EOSE
898
+ responses retain their normal cache behavior.
899
+
900
+ `getLatestEventsByPubkey` accepts `excludeRelaysByPubkey` (object or Map) for both
901
+ passes. `getRelaysByPubkey` accepts `excludeRelays` for seed discovery; pass it
902
+ through `relayListOptions` when using the two-pass query. Different exclusion
903
+ sets do not share in-flight discovery that could contact a refused relay.
904
+
905
+ `isRetryableRelayFailure(error)` identifies connection/transport/timeouts and
906
+ leading `rate-limited:`/`error:` relay failures. `isReplaceableRelayFailure(error)`
907
+ also admits relay policy refusals (`blocked:`, `restricted:`, `auth-required:`,
908
+ `pow:`) for routing to alternatives, preserving the messenger's existing policy.
909
+ Local authentication denials, invalid events and unknown errors qualify for
910
+ neither policy. Predicates classify individual native errors; callers decide
911
+ how to handle mixed per-relay outcomes and application-specific aggregates.
912
+ Existing rate-limit parsing preserves numeric positive `retry_after` seconds as
913
+ `retryAfterMs` and absolute `retryAt`, bounded to five minutes. Schedulers must
914
+ use the later of their backoff deadline and `retryAt`, not add the two delays.
915
+
883
916
  ## Binary encodings
884
917
 
885
918
  Base16, Base36, Base62, Base64/Base64URL, and Base93 helpers are available
@@ -991,7 +1024,10 @@ messages, history and files while preserving automatic NIP-65 updates. Successfu
991
1024
  latency, and short-lived channel/recipient preferences avoid recently refused
992
1025
  relays without changing receive subscriptions. See the
993
1026
  [session coordinator](private-messenger/session/README.md) for deferred offline
994
- attempts and final failure notifications.
1027
+ attempts and final failure notifications. Sends attempted while the messenger is
1028
+ paused or otherwise unavailable now stay pending in the outbox and retry on a
1029
+ bounded session backoff (1–30 seconds, 20% jitter) instead of surfacing a
1030
+ user-facing send error.
995
1031
 
996
1032
 
997
1033
  ### Relay backpressure and bounded synchronization (0.11.7)
@@ -1030,10 +1066,12 @@ readiness on the replacement. Callbacks and completions are generation-fenced.
1030
1066
  Equal watch settings cannot keep a dead subscription registered as active.
1031
1067
 
1032
1068
  Overflow recovery keeps one timer per relay with exponential 1–30 second
1033
- backoff and 20% jitter, reset after a stable minute. Other relays remain live.
1034
- Removal, pause and close cancel pending work. Fragment storage stays scoped to
1035
- the same session across replacement subscriptions. Permanent refusals remain
1036
- reported and do not trigger this automatic retry loop.
1069
+ backoff and 20% jitter, reset after a stable minute. Relay-pool capacity closes
1070
+ (1013, "relay pool capacity") use the same relay backoff instead of stopping
1071
+ until an explicit rebuild, because the launcher pool bucket budget is transient.
1072
+ Other relays remain live. Removal, pause and close cancel pending work. Fragment
1073
+ storage stays scoped to the same session across replacement subscriptions.
1074
+ Permanent refusals remain reported and do not trigger this automatic retry loop.
1037
1075
 
1038
1076
  PrivateMessenger persists each interrupted interval before reopening, then
1039
1077
  uses paged `fetchHistory` to repair the gap while listening for new messages.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "libp2r2p",
3
- "version": "0.11.12",
3
+ "version": "0.11.14",
4
4
  "description": "Peer-to-relay-to-peer",
5
5
  "keywords": [
6
6
  "p2r2p",
@@ -21,6 +21,18 @@ function uniq (values) {
21
21
  return [...new Set((values || []).filter(Boolean))]
22
22
  }
23
23
 
24
+ // The launcher's shared relay pool closes anonymous members with 1013 when its
25
+ // per-relay buckets are all busy. That is a transient pool condition, not a
26
+ // permanent relay rejection, so the shared subscription must keep the normal
27
+ // relay backoff instead of stopping until something else re-opens it.
28
+ function isRelayPoolCapacity (error) {
29
+ for (let current = error, depth = 0; current && depth < 3; current = current.cause, depth++) {
30
+ if (current.closeCode === 1013 || current.code === 'RELAY_POOL_CAPACITY') return true
31
+ if (/relay pool capacity/i.test(current.message || '')) return true
32
+ }
33
+ return false
34
+ }
35
+
24
36
  function normalizeDeletionPubkey (deletionPubkey) {
25
37
  if (deletionPubkey === undefined) return undefined
26
38
  if (typeof deletionPubkey !== 'string' || !HEX_PUBKEY.test(deletionPubkey)) {
@@ -360,8 +372,9 @@ export function createPrivateMessageSession ({ _setTimeout = setTimeout, _clearT
360
372
  delay: entry.readyAt !== undefined && _now() - entry.readyAt >= 60000 ? 1000 : (previous?.delay || entry.retryDelay || 1000),
361
373
  notified: new Map(), timer: null
362
374
  }
363
- if (/^(auth-required:|restricted:|blocked:|invalid:|pow:)/.test(error.message || '') || error.name === 'ValidationError' ||
364
- (error.code !== 'RELAY_LIVE_BUFFER_FULL' && !previous)) {
375
+ const permanent = /^(auth-required:|restricted:|blocked:|invalid:|pow:)/.test(error.message || '') || error.name === 'ValidationError'
376
+ const retryable = error.code === 'RELAY_LIVE_BUFFER_FULL' || isRelayPoolCapacity(error) || Boolean(previous)
377
+ if (permanent || !retryable) {
365
378
  cancelRelayRecovery(relay)
366
379
  notifyInterruption(relay, recovery).catch(error => reportSubscriptionError(entry.channels, error))
367
380
  return
@@ -1,19 +1,11 @@
1
1
  import { normalizeRelayUrl } from '../../url/index.js'
2
2
  import { ValidationError } from '../../error/index.js'
3
+ import { isReplaceableRelayFailure } from '../../relay/helpers/failure.js'
3
4
 
4
- const REJECTION_PREFIXES = new Set(['blocked', 'restricted', 'auth-required', 'pow', 'rate-limited', 'error'])
5
- const TRANSPORT_CATEGORIES = new Set(['connection', 'transport', 'timeout'])
6
5
  const EXCLUSION_MS = 5 * 60 * 1000
7
6
  const normalized = relay => { try { return normalizeRelayUrl(relay) } catch { return '' } }
8
7
 
9
- // Only machine-readable relay policy/server failures qualify. Invalid events,
10
- // local signer denials and unknown prose must not rotate relays.
11
- export function isReplaceableRelayFailure (error) {
12
- if (error?.name === 'Nip42AuthenticationError') return false
13
- if (TRANSPORT_CATEGORIES.has(error?.category)) return true
14
- if (error?.category && error.category !== 'relay') return false
15
- return REJECTION_PREFIXES.has(/^([a-z-]+):/.exec(error?.message || '')?.[1])
16
- }
8
+ export { isReplaceableRelayFailure }
17
9
 
18
10
  export function normalizeFallbackRelays (value = []) {
19
11
  if (!Array.isArray(value)) throw new ValidationError('INVALID_FALLBACK_RELAYS')
@@ -102,6 +94,11 @@ export function createSendRelayRouting ({ peer, peers = [peer], relaysByPubkey,
102
94
  errors: reports.flatMap(report => report.errors || [])
103
95
  }))
104
96
  })
97
+ // A lifecycle change (signer/pause) during a batch is not a user abort:
98
+ // every failure summary stays retryable so the session can re-attempt it.
99
+ const finished = result => current()
100
+ ? finish(result)
101
+ : { ...finish(result), retryWhenAvailable: !signal?.aborted, retryWhenOnline: !signal?.aborted }
105
102
  let result = { success: false }
106
103
  while ((batch ||= next())) {
107
104
  signal?.throwIfAborted()
@@ -124,15 +121,15 @@ export function createSendRelayRouting ({ peer, peers = [peer], relaysByPubkey,
124
121
  const report = await settled
125
122
  const failed = rejected(report)
126
123
  errors.push(...(report?.errors || []))
127
- if (!failed.length || errors.some(item => !isReplaceableRelayFailure(item.reason))) return finish(result)
128
- if (!next()) { exclusions.clear(); return finish(result) }
129
- if (!await online()) return { ...finish(result), retryWhenAvailable: !signal?.aborted, retryWhenOnline: current() }
124
+ if (!failed.length || errors.some(item => !isReplaceableRelayFailure(item.reason))) return finished(result)
125
+ if (!next()) { exclusions.clear(); return finished(result) }
126
+ if (!await online()) return { ...finish(result), retryWhenAvailable: !signal?.aborted, retryWhenOnline: !signal?.aborted }
130
127
  remember(failed)
131
128
  }
132
- if (!current()) return { ...finish(result), retryWhenAvailable: !signal?.aborted, retryWhenOnline: false }
129
+ if (!current()) return finished(result)
133
130
  batch = null
134
131
  }
135
- return finish(result)
132
+ return finished(result)
136
133
  }
137
134
 
138
135
  return {
@@ -40,6 +40,14 @@ or cancelled/closed sends. Apps own route visibility and user-intent filtering;
40
40
  automatic retries may also fail. Callbacks must not throw. Errors are not copied
41
41
  into persisted outbox records.
42
42
 
43
+ Sends that only failed because the messenger was paused or otherwise
44
+ unavailable (`PRIVATE_MESSENGER_PAUSED`, `CHAT_UNAVAILABLE`) stay pending
45
+ without `onSendError`. The session re-pumps them on a bounded backoff
46
+ (1–30 seconds with 20% jitter) in addition to availability/connectivity
47
+ changes, so a paused messenger recovers without a manual retry. The backoff
48
+ resets on a user `retry`, on `setAvailable(true)`, and once no retryable entry
49
+ remains.
50
+
43
51
  `enqueue({ peer, event, context, requiredFiles, deletion })` preserves prepared
44
52
  identities. Local message commits and remote stages have independent checkpoints.
45
53
  It sends original/thumbnail chunks before announcements, yields for small messages,
@@ -18,8 +18,20 @@ export function wireEvent (value, owner) {
18
18
  }
19
19
  // Relay rejection text must not be interpreted as a local signer denial.
20
20
  const retryable = error => error?.code === 'MESSAGE_NOT_PUBLISHED' || !/DENIED|PERMISSION|REVOKED|READ_ONLY|INVALID|BLOCKED|EXPIRED|NOT_IN_PERSONA/i.test(`${error?.code || ''} ${error?.message || ''}`)
21
+ const unavailableSend = /PRIVATE_MESSENGER_PAUSED|CHAT_UNAVAILABLE|MESSENGER_UNAVAILABLE/i
22
+ const isUnavailableSend = error => unavailableSend.test(`${error?.code || ''} ${error?.message || ''}`)
23
+ const SEND_RETRY_MIN_MS = 1000
24
+ const SEND_RETRY_MAX_MS = 30000
25
+ // A paused or unavailable messenger must keep the outbox entry retryable
26
+ // instead of surfacing a send failure the user has to retry by hand.
27
+ const unavailableError = (error, signal) => Object.assign(new Error(error?.message || 'CHAT_UNAVAILABLE'), {
28
+ code: error?.code,
29
+ cause: error,
30
+ retryWhenAvailable: !signal?.aborted,
31
+ retryWhenOnline: false
32
+ })
21
33
 
22
- export function createPrivateMessageSession ({ owner, signer, eventStore, messageStorage, chunkStorage, recoveryStorage, fallbackRelays = [], mode = 'seeder', seedersForPeer = peer => [peer], allowedKinds = [5, 9, 1063, 34601], useContentKeys = true, onMedia = () => {}, onOutbox = () => {}, onError = () => {}, onSendError = () => {}, Messenger = createPrivateMessenger, openOutbox = createChatOutbox, FileTransfer = createPrivateFileTransfer, _onOnline = onOnline, _getIykcProofs = getIykcProofs, openDownloads = options => createChatOutbox({ ...options, namespace: 'downloads' }) }) {
34
+ export function createPrivateMessageSession ({ owner, signer, eventStore, messageStorage, chunkStorage, recoveryStorage, fallbackRelays = [], mode = 'seeder', seedersForPeer = peer => [peer], allowedKinds = [5, 9, 1063, 34601], useContentKeys = true, onMedia = () => {}, onOutbox = () => {}, onError = () => {}, onSendError = () => {}, Messenger = createPrivateMessenger, openOutbox = createChatOutbox, FileTransfer = createPrivateFileTransfer, _onOnline = onOnline, _getIykcProofs = getIykcProofs, _setTimeout = setTimeout, _clearTimeout = clearTimeout, _random = Math.random, openDownloads = options => createChatOutbox({ ...options, namespace: 'downloads' }) }) {
23
35
  fallbackRelays = normalizeFallbackRelays(fallbackRelays)
24
36
  const reportedErrors = new WeakSet()
25
37
  const reportError = error => {
@@ -40,6 +52,8 @@ export function createPrivateMessageSession ({ owner, signer, eventStore, messag
40
52
  const sendControllers = new Map()
41
53
  const offlineSends = new Set()
42
54
  let stopOnline
55
+ let sendRetryTimer
56
+ let sendRetryDelay = SEND_RETRY_MIN_MS
43
57
  let messenger
44
58
  let fileTransfers
45
59
  let downloads
@@ -60,6 +74,7 @@ export function createPrivateMessageSession ({ owner, signer, eventStore, messag
60
74
  let lifecycle = 0
61
75
  let configuring = Promise.resolve()
62
76
  let sending
77
+ let repump
63
78
  let draining
64
79
  const idleWaiters = new Set()
65
80
  const settled = () => { if (!sending && !draining) { for (const resolve of idleWaiters) resolve(); idleWaiters.clear() } }
@@ -179,9 +194,15 @@ export function createPrivateMessageSession ({ owner, signer, eventStore, messag
179
194
  }
180
195
  async function publish (peer, event, signal) {
181
196
  if (deniedPeers.has(peer)) throw new Error('PERMISSION_DENIED')
182
- if (!available || !peers.has(peer) || !channels.has(peer)) throw new Error('CHAT_UNAVAILABLE')
197
+ if (!available || !peers.has(peer) || !channels.has(peer)) throw unavailableError(new Error('CHAT_UNAVAILABLE'), signal)
183
198
  const options = { channelPubkey: channels.get(peer)?.pubkey, receiverPubkeys: [peer], signal }
184
- const report = event.sig ? await messenger.broadcastEvent({ ...options, event }) : await messenger.broadcastRumor({ ...options, rumor: event })
199
+ let report
200
+ try {
201
+ report = event.sig ? await messenger.broadcastEvent({ ...options, event }) : await messenger.broadcastRumor({ ...options, rumor: event })
202
+ } catch (error) {
203
+ if (isUnavailableSend(error)) throw unavailableError(error, signal)
204
+ throw error
205
+ }
185
206
  // Published is relay acceptance, never a peer receipt.
186
207
  await assertMessagePublished(report, event)
187
208
  }
@@ -245,7 +266,10 @@ export function createPrivateMessageSession ({ owner, signer, eventStore, messag
245
266
  await storage.remove(entry.id)
246
267
  entries.delete(entry.id)
247
268
  emit()
248
- } catch (error) {
269
+ } catch (cause) {
270
+ // Pauses can surface at any stage (text publication, file authorization
271
+ // or chunk delivery). Normalize all of them into the retryable path.
272
+ const error = cause?.retryWhenAvailable ? cause : (isUnavailableSend(cause) ? unavailableError(cause, controller.signal) : cause)
249
273
  if (cancelled.has(entry.id)) { await storage.remove(entry.id); entries.delete(entry.id) } else {
250
274
  entry.status = error.retryWhenAvailable ? 'pending' : 'error'; entry.failed = true; entry.retryable = retryable(error)
251
275
  await storage.put(entry, { existing: true }).catch(reportError)
@@ -253,12 +277,42 @@ export function createPrivateMessageSession ({ owner, signer, eventStore, messag
253
277
  // The failed wire event may be a quote or file chunk. Report the
254
278
  // owning outbox item separately, without mutating the native error.
255
279
  if (error.retryWhenOnline) offlineSends.add(entry.id)
280
+ if (error.retryWhenAvailable) scheduleSendRetry()
256
281
  if (!closed && !entry.deletion && !cancelled.has(entry.id) && !error.retryWhenAvailable) onSendError(error, { id: entry.id, peer: entry.peer })
257
282
  }
258
283
  emit()
259
284
  } finally { sendControllers.delete(entry.id) }
260
285
  }
261
286
  function stopOnlineRetry () { stopOnline?.(); stopOnline = undefined }
287
+ // Entries that failed only because the messenger was paused/unavailable do
288
+ // not receive a connectivity transition, so they own a bounded backoff timer
289
+ // instead of waiting for the next account-state change.
290
+ function clearSendRetry () {
291
+ if (sendRetryTimer) _clearTimeout(sendRetryTimer)
292
+ sendRetryTimer = undefined
293
+ }
294
+ function resetSendRetry () {
295
+ clearSendRetry()
296
+ sendRetryDelay = SEND_RETRY_MIN_MS
297
+ }
298
+ const pendingRetryable = entry => Boolean(entry.failed && !cancelled.has(entry.id) && (entry.retryable || entry.status === 'pending'))
299
+ function scheduleSendRetry () {
300
+ if (closed || !available || sendRetryTimer) return
301
+ const delay = Math.min(SEND_RETRY_MAX_MS, sendRetryDelay * (0.8 + _random() * 0.4))
302
+ sendRetryDelay = Math.min(SEND_RETRY_MAX_MS, sendRetryDelay * 2)
303
+ sendRetryTimer = _setTimeout(() => {
304
+ sendRetryTimer = undefined
305
+ if (closed || !available) return
306
+ let retried = false
307
+ for (const entry of entries.values()) {
308
+ if (!pendingRetryable(entry)) continue
309
+ entry.failed = false
310
+ retried = true
311
+ }
312
+ if (retried) pump()
313
+ }, delay)
314
+ sendRetryTimer.unref?.()
315
+ }
262
316
  function watchOfflineSends () {
263
317
  if (closed || !available || !offlineSends.size) { stopOnlineRetry(); return }
264
318
  if (stopOnline) return
@@ -276,15 +330,26 @@ export function createPrivateMessageSession ({ owner, signer, eventStore, messag
276
330
  })
277
331
  }
278
332
  async function pump () {
279
- if (sending || !available || closed || !messenger || !storage) return
333
+ if (!available || closed || !messenger || !storage) return
334
+ // A retry that lands while a pump is already running must not be skipped
335
+ // just because that pass already walked past the entry.
336
+ if (sending) { repump = true; return }
280
337
  sending = true
281
338
  try {
282
- for (const entry of entries.values()) {
283
- if (closed || !available) break
284
- if ((entry.peer !== owner && !peers.has(entry.peer)) || entry.failed || cancelled.has(entry.id)) continue
285
- await sendEntry(entry)
286
- }
287
- } finally { sending = false; watchOfflineSends(); settled() }
339
+ do {
340
+ repump = false
341
+ for (const entry of entries.values()) {
342
+ if (closed || !available) break
343
+ if ((entry.peer !== owner && !peers.has(entry.peer)) || entry.failed || cancelled.has(entry.id)) continue
344
+ await sendEntry(entry)
345
+ }
346
+ } while (repump && !closed && available)
347
+ } finally {
348
+ sending = false
349
+ watchOfflineSends()
350
+ if (![...entries.values()].some(pendingRetryable)) resetSendRetry()
351
+ settled()
352
+ }
288
353
  }
289
354
  const intentId = file => getEventHash({ kind: 0, pubkey: owner, created_at: 0, tags: [], content: `${file.peer}:${file.root}` })
290
355
  async function download (file, options = {}) {
@@ -359,10 +424,12 @@ export function createPrivateMessageSession ({ owner, signer, eventStore, messag
359
424
  async setAvailable (value) {
360
425
  available = value === true
361
426
  stopOnlineRetry(); offlineSends.clear()
427
+ if (available) resetSendRetry()
428
+ else clearSendRetry()
362
429
  lifecycle++
363
430
  if (!available) { await messenger?.pause('signer'); return }
364
431
  await ready()
365
- for (const entry of entries.values()) if (entry.retryable) entry.failed = false
432
+ for (const entry of entries.values()) if (pendingRetryable(entry)) entry.failed = false
366
433
  await messenger?.resume('storage')
367
434
  return schedule()
368
435
  },
@@ -382,7 +449,7 @@ export function createPrivateMessageSession ({ owner, signer, eventStore, messag
382
449
  offlineSends.delete(id); watchOfflineSends()
383
450
  await ready()
384
451
  const entry = entries.get(id)
385
- if (entry) { entry.failed = false; if (deniedPeers.delete(entry.peer)) await schedule(); return pump() }
452
+ if (entry) { entry.failed = false; resetSendRetry(); if (deniedPeers.delete(entry.peer)) await schedule(); return pump() }
386
453
  },
387
454
  async cancel (id) {
388
455
  cancelled.add(id)
@@ -396,6 +463,7 @@ export function createPrivateMessageSession ({ owner, signer, eventStore, messag
396
463
  async close () {
397
464
  closed = true; available = false; lifecycle++
398
465
  stopOnlineRetry(); offlineSends.clear(); activeDownloads.clear(); pendingPriorityRanges.clear()
466
+ clearSendRetry()
399
467
  for (const controller of sendControllers.values()) controller.abort()
400
468
  await Promise.allSettled([configuring, initialized])
401
469
  await messenger?.pause('closed')
@@ -0,0 +1,15 @@
1
+ const TRANSPORT_CATEGORIES = new Set(['connection', 'transport', 'timeout'])
2
+ const RETRY_PREFIXES = new Set(['rate-limited', 'error'])
3
+ const REPLACEMENT_PREFIXES = new Set(['blocked', 'restricted', 'auth-required', 'pow', ...RETRY_PREFIXES])
4
+
5
+ // Retry and replacement are different policies: a refusal can justify another
6
+ // relay without making the same operation retryable on the refusing relay.
7
+ function matches (error, prefixes) {
8
+ if (error?.name === 'Nip42AuthenticationError') return false
9
+ if (TRANSPORT_CATEGORIES.has(error?.category)) return true
10
+ if (error?.category && error.category !== 'relay') return false
11
+ return prefixes.has(/^([a-z-]+):/.exec(error?.message || '')?.[1])
12
+ }
13
+
14
+ export const isRetryableRelayFailure = error => matches(error, RETRY_PREFIXES)
15
+ export const isReplaceableRelayFailure = error => matches(error, REPLACEMENT_PREFIXES)
package/relay/index.js CHANGED
@@ -1,4 +1,5 @@
1
1
  export { freeRelays, nappRelays, seedRelays } from './constants/index.js'
2
+ export { isRetryableRelayFailure, isReplaceableRelayFailure } from './helpers/failure.js'
2
3
  export { pickRelaysForPubkeys } from './helpers/routing.js'
3
4
  export { RelayPool, relayPool } from './services/relay-pool.js'
4
5
  export { getLatestEventsByPubkey } from './services/events.js'
@@ -45,7 +45,7 @@ function getSelectedRelaysByPubkey (pubkeys, relayToAuthors) {
45
45
  }
46
46
 
47
47
  // Fetches batched events and keeps the newest event per address.
48
- async function fetchLatestEventsByRelay (relayToAuthors, { kinds, dTagsByPubkey, getEvents }) {
48
+ async function fetchLatestEventsByRelay (relayToAuthors, { kinds, dTagsByPubkey, getEvents, report, phase, signal }) {
49
49
  const requests = []
50
50
  for (const [relay, authors] of relayToAuthors) {
51
51
  const authorsByD = new Map()
@@ -57,8 +57,17 @@ async function fetchLatestEventsByRelay (relayToAuthors, { kinds, dTagsByPubkey,
57
57
  for (const [d, dAuthors] of authorsByD) {
58
58
  const filter = { kinds, authors: dAuthors }
59
59
  if (d) filter['#d'] = [d]
60
- requests.push(getEvents(filter, [relay])
61
- .then(response => ({ requested: new Set(dAuthors), events: (response.result || []).map(({ event }) => event) })))
60
+ requests.push(Promise.resolve().then(async () => {
61
+ signal?.throwIfAborted()
62
+ try {
63
+ const response = await getEvents(filter, [relay], signal ? { signal } : {})
64
+ report({ phase, authors: dAuthors, relays: response.relays || [], errors: response.errors || [] })
65
+ return { requested: new Set(dAuthors), events: (response.result || []).map(({ event }) => event) }
66
+ } catch (error) {
67
+ report({ phase, authors: dAuthors, relays: [{ relay, status: 'error', error }], error })
68
+ throw error
69
+ }
70
+ }))
62
71
  }
63
72
  }
64
73
 
@@ -95,11 +104,20 @@ export async function getLatestEventsByPubkey (pubkeys, {
95
104
  fallbackRelays = freeRelays.slice(0, DEFAULT_FALLBACK_RELAY_COUNT),
96
105
  relaysByPubkey,
97
106
  relayListOptions,
107
+ excludeRelaysByPubkey,
108
+ signal,
109
+ onQueryResult,
98
110
  _getRelaysByPubkey = getRelaysByPubkey,
99
111
  _getEvents = getEvents
100
112
  } = {}) {
113
+ signal?.throwIfAborted()
114
+ const requests = []
115
+ const report = request => { requests.push(request); onQueryResult?.(request) }
116
+ const excluded = pubkey => excludeRelaysByPubkey instanceof Map
117
+ ? excludeRelaysByPubkey.get(pubkey) || []
118
+ : excludeRelaysByPubkey?.[pubkey] || []
101
119
  const authors = [...new Set(pubkeys || [])].filter(Boolean)
102
- if (!authors.length) return { events: [], byPubkey: {}, relaysByPubkey: {} }
120
+ if (!authors.length) return { events: [], byPubkey: {}, relaysByPubkey: {}, requests }
103
121
  if (!Array.isArray(kinds) || kinds.length === 0) {
104
122
  throw new ValidationError('MISSING_EVENT_KINDS', { message: 'Missing kinds' })
105
123
  }
@@ -110,9 +128,16 @@ export async function getLatestEventsByPubkey (pubkeys, {
110
128
  if (missingRelayAuthors.length) {
111
129
  let discovered
112
130
  try {
113
- discovered = await _getRelaysByPubkey(missingRelayAuthors, relayListOptions)
131
+ discovered = await _getRelaysByPubkey(missingRelayAuthors, {
132
+ ...relayListOptions,
133
+ ...(signal ? { signal } : {}),
134
+ onQueryResult: request => { report({ ...request, phase: 'discovery' }); relayListOptions?.onQueryResult?.(request) }
135
+ })
114
136
  } catch (error) {
115
- console.error('Failed to discover publisher relays:', error)
137
+ signal?.throwIfAborted()
138
+ if (!requests.some(request => request.phase === 'discovery' && request.error === error)) {
139
+ report({ phase: 'discovery', authors: missingRelayAuthors, relays: [], error })
140
+ }
116
141
  discovered = Object.fromEntries(missingRelayAuthors.map(pubkey => [pubkey, { read: [], write: [] }]))
117
142
  }
118
143
  for (const pubkey of missingRelayAuthors) {
@@ -120,14 +145,17 @@ export async function getLatestEventsByPubkey (pubkeys, {
120
145
  }
121
146
  }
122
147
 
148
+ signal?.throwIfAborted()
123
149
  const primaryAuthors = authors.filter(pubkey => relaysByAuthor[pubkey]?.[type]?.length)
124
- const primaryRoutes = pickRelaysForPubkeys(primaryAuthors, relaysByAuthor, { maxPerPubkey, relayType })
150
+ const primaryRoutes = pickRelaysForPubkeys(primaryAuthors, relaysByAuthor, { maxPerPubkey, relayType, excludeRelaysByPubkey, emptyRelaysFallback: [] })
125
151
  const selectedRelays = getSelectedRelaysByPubkey(authors, primaryRoutes)
152
+ for (const pubkey of authors) for (const relay of excluded(pubkey)) selectedRelays[pubkey].add(relay)
126
153
  const latestByAddress = await fetchLatestEventsByRelay(primaryRoutes, {
127
154
  kinds,
128
155
  dTagsByPubkey,
129
- getEvents: _getEvents
156
+ getEvents: _getEvents, report, phase: 'primary', signal
130
157
  })
158
+ signal?.throwIfAborted()
131
159
  const foundPubkeys = new Set(Object.values(latestByAddress).map(event => event.pubkey))
132
160
  const missingAuthors = authors.filter(pubkey => !foundPubkeys.has(pubkey))
133
161
 
@@ -145,13 +173,14 @@ export async function getLatestEventsByPubkey (pubkeys, {
145
173
  await fetchLatestEventsByRelay(fallbackRoutes, {
146
174
  kinds,
147
175
  dTagsByPubkey,
148
- getEvents: _getEvents
176
+ getEvents: _getEvents, report, phase: 'fallback', signal
149
177
  })
150
178
  ))
151
179
  }
152
180
 
181
+ signal?.throwIfAborted()
153
182
  const events = Object.values(latestByAddress)
154
183
  const byPubkey = {}
155
184
  for (const event of events) byPubkey[event.pubkey] = event
156
- return { events, byPubkey, relaysByPubkey: relaysByAuthor }
185
+ return { events, byPubkey, relaysByPubkey: relaysByAuthor, requests }
157
186
  }
@@ -224,28 +224,27 @@ export function subscribeRelayListUpdates (pubkeys, {
224
224
  }
225
225
 
226
226
  async function loadMissingRelays (missingPubkeys, {
227
- getEvents,
228
- cacheMs,
229
- timeout = RELAY_LIST_QUERY_TIMEOUT_MS,
230
- timeoutAfterFirstEose = RELAY_LIST_QUERY_TIMEOUT_AFTER_FIRST_EOSE_MS,
231
- relayUrlPolicy,
232
- emptyRelaysFallback = freeRelays.slice(0, 2)
227
+ getEvents, cacheMs, timeout, timeoutAfterFirstEose, relayUrlPolicy,
228
+ emptyRelaysFallback, excludeRelays, signal
233
229
  }) {
234
- const { result: events } = await getEvents({
235
- kinds: [10002],
236
- authors: missingPubkeys,
237
- limit: missingPubkeys.length
238
- }, seedRelays, {
239
- timeout,
240
- timeoutAfterFirstEose
241
- })
242
-
230
+ let response
231
+ try {
232
+ response = await getEvents({ kinds: [10002], authors: missingPubkeys, limit: missingPubkeys.length },
233
+ seedRelays.filter(relay => !excludeRelays.includes(relay)), { timeout, timeoutAfterFirstEose, signal })
234
+ signal.throwIfAborted()
235
+ } catch (error) {
236
+ return { entries: {}, report: { authors: missingPubkeys, relays: [], error } }
237
+ }
238
+ const report = { authors: missingPubkeys, relays: response.relays || [], errors: response.errors || [] }
239
+ const complete = response.relays
240
+ ? response.relays.length > 0 && response.relays.every(item => item.status === 'eose')
241
+ : !response.errors?.length
243
242
  const latestByPubkey = {}
244
- for (const { event } of events || []) {
243
+ for (const { event } of response.result || []) {
245
244
  if (!missingPubkeys.includes(event.pubkey)) continue
246
245
  if (isNewerRelayListEvent(event, latestByPubkey[event.pubkey])) latestByPubkey[event.pubkey] = event
247
246
  }
248
-
247
+ const entries = {}
249
248
  for (const pubkey of missingPubkeys) {
250
249
  const fetchedEvent = latestByPubkey[pubkey]
251
250
  const cachedEvent = relayCacheEventByPubkey[pubkey] || null
@@ -253,9 +252,29 @@ async function loadMissingRelays (missingPubkeys, {
253
252
  const relays = event
254
253
  ? parseRelayListEvent(event, relayUrlPolicy)
255
254
  : { read: [...emptyRelaysFallback], write: [...emptyRelaysFallback] }
256
- setCachedRelays(pubkey, relays, event, cacheMs)
255
+ entries[pubkey] = { ...relays, event }
256
+ // A failed/incomplete lookup may use fallbacks for this call, but must not
257
+ // turn that failure into a forty-minute negative discovery cache entry.
258
+ if (fetchedEvent || complete) setCachedRelays(pubkey, relays, event, cacheMs)
257
259
  }
258
260
  pruneRelayCache()
261
+ return { entries, report }
262
+ }
263
+
264
+ // Each caller owns its wait, not another caller's shared discovery request.
265
+ function waitForDiscovery (record, signal) {
266
+ signal?.throwIfAborted()
267
+ record.users++
268
+ return new Promise((resolve, reject) => {
269
+ const abort = () => reject(signal.reason)
270
+ signal?.addEventListener('abort', abort, { once: true })
271
+ record.promise.then(resolve, reject).finally(() => signal?.removeEventListener('abort', abort))
272
+ }).finally(() => {
273
+ if (--record.users === 0 && !record.finished) {
274
+ record.controller.abort()
275
+ record.remove()
276
+ }
277
+ })
259
278
  }
260
279
 
261
280
  export async function getRelaysByPubkey (pubkeys, {
@@ -266,40 +285,48 @@ export async function getRelaysByPubkey (pubkeys, {
266
285
  timeout = RELAY_LIST_QUERY_TIMEOUT_MS,
267
286
  timeoutAfterFirstEose = RELAY_LIST_QUERY_TIMEOUT_AFTER_FIRST_EOSE_MS,
268
287
  relayUrlPolicy,
269
- emptyRelaysFallback = freeRelays.slice(0, 2)
288
+ emptyRelaysFallback = freeRelays.slice(0, 2),
289
+ excludeRelays = [],
290
+ signal,
291
+ onQueryResult
270
292
  } = {}) {
293
+ signal?.throwIfAborted()
271
294
  const pubkeyList = uniquePubkeys(pubkeys, { requireHex: _getEvents === getEvents })
272
295
  if (!pubkeyList.length) return {}
273
-
274
- const loadPubkeys = forceRefresh
275
- ? pubkeyList
276
- : pubkeyList.filter(pubkey => !hasCachedKey(relaysByPubkey, pubkey))
277
- const pubkeysToLoad = loadPubkeys.filter(pubkey => !relayRequestsByPubkey.has(pubkey))
296
+ // Different exclusions cannot share a request that contacts a refused relay.
297
+ const scope = JSON.stringify([...new Set(excludeRelays)].sort())
298
+ const key = pubkey => `${pubkey}:${scope}`
299
+ const loadPubkeys = forceRefresh ? pubkeyList : pubkeyList.filter(pubkey => !hasCachedKey(relaysByPubkey, pubkey))
300
+ const pubkeysToLoad = loadPubkeys.filter(pubkey => !relayRequestsByPubkey.has(key(pubkey)))
278
301
  if (pubkeysToLoad.length) {
279
- const request = loadMissingRelays(pubkeysToLoad, {
280
- getEvents: _getEvents,
281
- cacheMs,
282
- timeout,
283
- timeoutAfterFirstEose,
284
- relayUrlPolicy,
285
- emptyRelaysFallback
286
- }).finally(() => {
302
+ const record = { controller: new AbortController(), users: 0, finished: false }
303
+ record.remove = () => {
287
304
  for (const pubkey of pubkeysToLoad) {
288
- if (relayRequestsByPubkey.get(pubkey) === request) relayRequestsByPubkey.delete(pubkey)
305
+ if (relayRequestsByPubkey.get(key(pubkey)) === record) relayRequestsByPubkey.delete(key(pubkey))
289
306
  }
290
- })
291
- for (const pubkey of pubkeysToLoad) relayRequestsByPubkey.set(pubkey, request)
307
+ }
308
+ record.promise = loadMissingRelays(pubkeysToLoad, {
309
+ getEvents: _getEvents, cacheMs, timeout, timeoutAfterFirstEose, relayUrlPolicy,
310
+ emptyRelaysFallback, excludeRelays, signal: record.controller.signal
311
+ }).finally(() => { record.finished = true; record.remove() })
312
+ for (const pubkey of pubkeysToLoad) relayRequestsByPubkey.set(key(pubkey), record)
292
313
  }
293
-
294
- await Promise.all([...new Set(
295
- loadPubkeys.map(pubkey => relayRequestsByPubkey.get(pubkey)).filter(Boolean)
296
- )])
297
-
298
- return Object.fromEntries(pubkeyList
299
- .filter(pubkey => hasCachedKey(relaysByPubkey, pubkey))
300
- .map(pubkey => {
301
- const entry = cloneRelays(relaysByPubkey[pubkey])
302
- if (includeEvents) entry.event = cloneRelayListEvent(relayCacheEventByPubkey[pubkey])
303
- return [pubkey, entry]
304
- }))
314
+ const entries = {}
315
+ const records = [...new Set(loadPubkeys.map(pubkey => relayRequestsByPubkey.get(key(pubkey))).filter(Boolean))]
316
+ await Promise.all(records.map(async record => {
317
+ const result = await waitForDiscovery(record, signal)
318
+ signal?.throwIfAborted()
319
+ // Deliver the same native diagnostics to every interested caller, including
320
+ // callers that joined an already running query.
321
+ onQueryResult?.(result.report)
322
+ if (result.report.error) throw result.report.error
323
+ Object.assign(entries, result.entries)
324
+ }))
325
+ signal?.throwIfAborted()
326
+ return Object.fromEntries(pubkeyList.map(pubkey => {
327
+ const source = hasCachedKey(relaysByPubkey, pubkey) ? relaysByPubkey[pubkey] : entries[pubkey]
328
+ const entry = cloneRelays(source || { read: emptyRelaysFallback, write: emptyRelaysFallback })
329
+ if (includeEvents) entry.event = cloneRelayListEvent(relayCacheEventByPubkey[pubkey] || entries[pubkey]?.event)
330
+ return [pubkey, entry]
331
+ }))
305
332
  }