libp2r2p 0.11.12 → 0.11.14
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +44 -6
- package/package.json +1 -1
- package/private-message/index.js +15 -2
- package/private-messenger/helpers/send-routing.js +12 -15
- package/private-messenger/session/README.md +8 -0
- package/private-messenger/session/index.js +81 -13
- package/relay/helpers/failure.js +15 -0
- package/relay/index.js +1 -0
- package/relay/services/events.js +39 -10
- package/relay/services/query.js +74 -47
package/README.md
CHANGED
|
@@ -877,9 +877,42 @@ the first pass routes through up to `maxPerPubkey` shared relays, and missing
|
|
|
877
877
|
authors are retried on every remaining relay plus `fallbackRelays` (default:
|
|
878
878
|
the first three `freeRelays`), excluding what was already queried. It accepts
|
|
879
879
|
`relaysByPubkey` to reuse a previous discovery (only missing pubkeys are then
|
|
880
|
-
discovered) and returns `{ events, byPubkey, relaysByPubkey }` so the merged
|
|
880
|
+
discovered) and returns `{ events, byPubkey, relaysByPubkey, requests }` so the merged
|
|
881
881
|
relay map can be passed back on later calls.
|
|
882
882
|
|
|
883
|
+
|
|
884
|
+
Read diagnostics preserve native failures rather than treating an empty result
|
|
885
|
+
as proof that every relay completed successfully. `requests` contains records
|
|
886
|
+
`{ phase, authors, relays, errors?, error? }`: phase is `discovery`, `primary` or
|
|
887
|
+
`fallback`; `relays` uses the pool's `{ relay, status, error? }` outcomes. An
|
|
888
|
+
operation-wide exception is retained as `error`. Events remain in the existing
|
|
889
|
+
result fields, never in diagnostic records. `onQueryResult(record)` receives each
|
|
890
|
+
record as the underlying query settles, including before a later cancellation.
|
|
891
|
+
|
|
892
|
+
Both discovery and latest-event queries accept `signal`. Cancelling one consumer
|
|
893
|
+
of shared NIP-65 discovery releases only its interest; the underlying query is
|
|
894
|
+
aborted when its last consumer leaves. Discovery's optional `onQueryResult`
|
|
895
|
+
receives the same native report for every joining caller. Incomplete/failed
|
|
896
|
+
lookups can use fallback relays for that call but never negatively cache a
|
|
897
|
+
missing relay list for forty minutes. Actual lists and completely empty EOSE
|
|
898
|
+
responses retain their normal cache behavior.
|
|
899
|
+
|
|
900
|
+
`getLatestEventsByPubkey` accepts `excludeRelaysByPubkey` (object or Map) for both
|
|
901
|
+
passes. `getRelaysByPubkey` accepts `excludeRelays` for seed discovery; pass it
|
|
902
|
+
through `relayListOptions` when using the two-pass query. Different exclusion
|
|
903
|
+
sets do not share in-flight discovery that could contact a refused relay.
|
|
904
|
+
|
|
905
|
+
`isRetryableRelayFailure(error)` identifies connection/transport/timeouts and
|
|
906
|
+
leading `rate-limited:`/`error:` relay failures. `isReplaceableRelayFailure(error)`
|
|
907
|
+
also admits relay policy refusals (`blocked:`, `restricted:`, `auth-required:`,
|
|
908
|
+
`pow:`) for routing to alternatives, preserving the messenger's existing policy.
|
|
909
|
+
Local authentication denials, invalid events and unknown errors qualify for
|
|
910
|
+
neither policy. Predicates classify individual native errors; callers decide
|
|
911
|
+
how to handle mixed per-relay outcomes and application-specific aggregates.
|
|
912
|
+
Existing rate-limit parsing preserves numeric positive `retry_after` seconds as
|
|
913
|
+
`retryAfterMs` and absolute `retryAt`, bounded to five minutes. Schedulers must
|
|
914
|
+
use the later of their backoff deadline and `retryAt`, not add the two delays.
|
|
915
|
+
|
|
883
916
|
## Binary encodings
|
|
884
917
|
|
|
885
918
|
Base16, Base36, Base62, Base64/Base64URL, and Base93 helpers are available
|
|
@@ -991,7 +1024,10 @@ messages, history and files while preserving automatic NIP-65 updates. Successfu
|
|
|
991
1024
|
latency, and short-lived channel/recipient preferences avoid recently refused
|
|
992
1025
|
relays without changing receive subscriptions. See the
|
|
993
1026
|
[session coordinator](private-messenger/session/README.md) for deferred offline
|
|
994
|
-
attempts and final failure notifications.
|
|
1027
|
+
attempts and final failure notifications. Sends attempted while the messenger is
|
|
1028
|
+
paused or otherwise unavailable now stay pending in the outbox and retry on a
|
|
1029
|
+
bounded session backoff (1–30 seconds, 20% jitter) instead of surfacing a
|
|
1030
|
+
user-facing send error.
|
|
995
1031
|
|
|
996
1032
|
|
|
997
1033
|
### Relay backpressure and bounded synchronization (0.11.7)
|
|
@@ -1030,10 +1066,12 @@ readiness on the replacement. Callbacks and completions are generation-fenced.
|
|
|
1030
1066
|
Equal watch settings cannot keep a dead subscription registered as active.
|
|
1031
1067
|
|
|
1032
1068
|
Overflow recovery keeps one timer per relay with exponential 1–30 second
|
|
1033
|
-
backoff and 20% jitter, reset after a stable minute.
|
|
1034
|
-
|
|
1035
|
-
|
|
1036
|
-
|
|
1069
|
+
backoff and 20% jitter, reset after a stable minute. Relay-pool capacity closes
|
|
1070
|
+
(1013, "relay pool capacity") use the same relay backoff instead of stopping
|
|
1071
|
+
until an explicit rebuild, because the launcher pool bucket budget is transient.
|
|
1072
|
+
Other relays remain live. Removal, pause and close cancel pending work. Fragment
|
|
1073
|
+
storage stays scoped to the same session across replacement subscriptions.
|
|
1074
|
+
Permanent refusals remain reported and do not trigger this automatic retry loop.
|
|
1037
1075
|
|
|
1038
1076
|
PrivateMessenger persists each interrupted interval before reopening, then
|
|
1039
1077
|
uses paged `fetchHistory` to repair the gap while listening for new messages.
|
package/package.json
CHANGED
package/private-message/index.js
CHANGED
|
@@ -21,6 +21,18 @@ function uniq (values) {
|
|
|
21
21
|
return [...new Set((values || []).filter(Boolean))]
|
|
22
22
|
}
|
|
23
23
|
|
|
24
|
+
// The launcher's shared relay pool closes anonymous members with 1013 when its
|
|
25
|
+
// per-relay buckets are all busy. That is a transient pool condition, not a
|
|
26
|
+
// permanent relay rejection, so the shared subscription must keep the normal
|
|
27
|
+
// relay backoff instead of stopping until something else re-opens it.
|
|
28
|
+
function isRelayPoolCapacity (error) {
|
|
29
|
+
for (let current = error, depth = 0; current && depth < 3; current = current.cause, depth++) {
|
|
30
|
+
if (current.closeCode === 1013 || current.code === 'RELAY_POOL_CAPACITY') return true
|
|
31
|
+
if (/relay pool capacity/i.test(current.message || '')) return true
|
|
32
|
+
}
|
|
33
|
+
return false
|
|
34
|
+
}
|
|
35
|
+
|
|
24
36
|
function normalizeDeletionPubkey (deletionPubkey) {
|
|
25
37
|
if (deletionPubkey === undefined) return undefined
|
|
26
38
|
if (typeof deletionPubkey !== 'string' || !HEX_PUBKEY.test(deletionPubkey)) {
|
|
@@ -360,8 +372,9 @@ export function createPrivateMessageSession ({ _setTimeout = setTimeout, _clearT
|
|
|
360
372
|
delay: entry.readyAt !== undefined && _now() - entry.readyAt >= 60000 ? 1000 : (previous?.delay || entry.retryDelay || 1000),
|
|
361
373
|
notified: new Map(), timer: null
|
|
362
374
|
}
|
|
363
|
-
|
|
364
|
-
|
|
375
|
+
const permanent = /^(auth-required:|restricted:|blocked:|invalid:|pow:)/.test(error.message || '') || error.name === 'ValidationError'
|
|
376
|
+
const retryable = error.code === 'RELAY_LIVE_BUFFER_FULL' || isRelayPoolCapacity(error) || Boolean(previous)
|
|
377
|
+
if (permanent || !retryable) {
|
|
365
378
|
cancelRelayRecovery(relay)
|
|
366
379
|
notifyInterruption(relay, recovery).catch(error => reportSubscriptionError(entry.channels, error))
|
|
367
380
|
return
|
|
@@ -1,19 +1,11 @@
|
|
|
1
1
|
import { normalizeRelayUrl } from '../../url/index.js'
|
|
2
2
|
import { ValidationError } from '../../error/index.js'
|
|
3
|
+
import { isReplaceableRelayFailure } from '../../relay/helpers/failure.js'
|
|
3
4
|
|
|
4
|
-
const REJECTION_PREFIXES = new Set(['blocked', 'restricted', 'auth-required', 'pow', 'rate-limited', 'error'])
|
|
5
|
-
const TRANSPORT_CATEGORIES = new Set(['connection', 'transport', 'timeout'])
|
|
6
5
|
const EXCLUSION_MS = 5 * 60 * 1000
|
|
7
6
|
const normalized = relay => { try { return normalizeRelayUrl(relay) } catch { return '' } }
|
|
8
7
|
|
|
9
|
-
|
|
10
|
-
// local signer denials and unknown prose must not rotate relays.
|
|
11
|
-
export function isReplaceableRelayFailure (error) {
|
|
12
|
-
if (error?.name === 'Nip42AuthenticationError') return false
|
|
13
|
-
if (TRANSPORT_CATEGORIES.has(error?.category)) return true
|
|
14
|
-
if (error?.category && error.category !== 'relay') return false
|
|
15
|
-
return REJECTION_PREFIXES.has(/^([a-z-]+):/.exec(error?.message || '')?.[1])
|
|
16
|
-
}
|
|
8
|
+
export { isReplaceableRelayFailure }
|
|
17
9
|
|
|
18
10
|
export function normalizeFallbackRelays (value = []) {
|
|
19
11
|
if (!Array.isArray(value)) throw new ValidationError('INVALID_FALLBACK_RELAYS')
|
|
@@ -102,6 +94,11 @@ export function createSendRelayRouting ({ peer, peers = [peer], relaysByPubkey,
|
|
|
102
94
|
errors: reports.flatMap(report => report.errors || [])
|
|
103
95
|
}))
|
|
104
96
|
})
|
|
97
|
+
// A lifecycle change (signer/pause) during a batch is not a user abort:
|
|
98
|
+
// every failure summary stays retryable so the session can re-attempt it.
|
|
99
|
+
const finished = result => current()
|
|
100
|
+
? finish(result)
|
|
101
|
+
: { ...finish(result), retryWhenAvailable: !signal?.aborted, retryWhenOnline: !signal?.aborted }
|
|
105
102
|
let result = { success: false }
|
|
106
103
|
while ((batch ||= next())) {
|
|
107
104
|
signal?.throwIfAborted()
|
|
@@ -124,15 +121,15 @@ export function createSendRelayRouting ({ peer, peers = [peer], relaysByPubkey,
|
|
|
124
121
|
const report = await settled
|
|
125
122
|
const failed = rejected(report)
|
|
126
123
|
errors.push(...(report?.errors || []))
|
|
127
|
-
if (!failed.length || errors.some(item => !isReplaceableRelayFailure(item.reason))) return
|
|
128
|
-
if (!next()) { exclusions.clear(); return
|
|
129
|
-
if (!await online()) return { ...finish(result), retryWhenAvailable: !signal?.aborted, retryWhenOnline:
|
|
124
|
+
if (!failed.length || errors.some(item => !isReplaceableRelayFailure(item.reason))) return finished(result)
|
|
125
|
+
if (!next()) { exclusions.clear(); return finished(result) }
|
|
126
|
+
if (!await online()) return { ...finish(result), retryWhenAvailable: !signal?.aborted, retryWhenOnline: !signal?.aborted }
|
|
130
127
|
remember(failed)
|
|
131
128
|
}
|
|
132
|
-
if (!current()) return
|
|
129
|
+
if (!current()) return finished(result)
|
|
133
130
|
batch = null
|
|
134
131
|
}
|
|
135
|
-
return
|
|
132
|
+
return finished(result)
|
|
136
133
|
}
|
|
137
134
|
|
|
138
135
|
return {
|
|
@@ -40,6 +40,14 @@ or cancelled/closed sends. Apps own route visibility and user-intent filtering;
|
|
|
40
40
|
automatic retries may also fail. Callbacks must not throw. Errors are not copied
|
|
41
41
|
into persisted outbox records.
|
|
42
42
|
|
|
43
|
+
Sends that only failed because the messenger was paused or otherwise
|
|
44
|
+
unavailable (`PRIVATE_MESSENGER_PAUSED`, `CHAT_UNAVAILABLE`) stay pending
|
|
45
|
+
without `onSendError`. The session re-pumps them on a bounded backoff
|
|
46
|
+
(1–30 seconds with 20% jitter) in addition to availability/connectivity
|
|
47
|
+
changes, so a paused messenger recovers without a manual retry. The backoff
|
|
48
|
+
resets on a user `retry`, on `setAvailable(true)`, and once no retryable entry
|
|
49
|
+
remains.
|
|
50
|
+
|
|
43
51
|
`enqueue({ peer, event, context, requiredFiles, deletion })` preserves prepared
|
|
44
52
|
identities. Local message commits and remote stages have independent checkpoints.
|
|
45
53
|
It sends original/thumbnail chunks before announcements, yields for small messages,
|
|
@@ -18,8 +18,20 @@ export function wireEvent (value, owner) {
|
|
|
18
18
|
}
|
|
19
19
|
// Relay rejection text must not be interpreted as a local signer denial.
|
|
20
20
|
const retryable = error => error?.code === 'MESSAGE_NOT_PUBLISHED' || !/DENIED|PERMISSION|REVOKED|READ_ONLY|INVALID|BLOCKED|EXPIRED|NOT_IN_PERSONA/i.test(`${error?.code || ''} ${error?.message || ''}`)
|
|
21
|
+
const unavailableSend = /PRIVATE_MESSENGER_PAUSED|CHAT_UNAVAILABLE|MESSENGER_UNAVAILABLE/i
|
|
22
|
+
const isUnavailableSend = error => unavailableSend.test(`${error?.code || ''} ${error?.message || ''}`)
|
|
23
|
+
const SEND_RETRY_MIN_MS = 1000
|
|
24
|
+
const SEND_RETRY_MAX_MS = 30000
|
|
25
|
+
// A paused or unavailable messenger must keep the outbox entry retryable
|
|
26
|
+
// instead of surfacing a send failure the user has to retry by hand.
|
|
27
|
+
const unavailableError = (error, signal) => Object.assign(new Error(error?.message || 'CHAT_UNAVAILABLE'), {
|
|
28
|
+
code: error?.code,
|
|
29
|
+
cause: error,
|
|
30
|
+
retryWhenAvailable: !signal?.aborted,
|
|
31
|
+
retryWhenOnline: false
|
|
32
|
+
})
|
|
21
33
|
|
|
22
|
-
export function createPrivateMessageSession ({ owner, signer, eventStore, messageStorage, chunkStorage, recoveryStorage, fallbackRelays = [], mode = 'seeder', seedersForPeer = peer => [peer], allowedKinds = [5, 9, 1063, 34601], useContentKeys = true, onMedia = () => {}, onOutbox = () => {}, onError = () => {}, onSendError = () => {}, Messenger = createPrivateMessenger, openOutbox = createChatOutbox, FileTransfer = createPrivateFileTransfer, _onOnline = onOnline, _getIykcProofs = getIykcProofs, openDownloads = options => createChatOutbox({ ...options, namespace: 'downloads' }) }) {
|
|
34
|
+
export function createPrivateMessageSession ({ owner, signer, eventStore, messageStorage, chunkStorage, recoveryStorage, fallbackRelays = [], mode = 'seeder', seedersForPeer = peer => [peer], allowedKinds = [5, 9, 1063, 34601], useContentKeys = true, onMedia = () => {}, onOutbox = () => {}, onError = () => {}, onSendError = () => {}, Messenger = createPrivateMessenger, openOutbox = createChatOutbox, FileTransfer = createPrivateFileTransfer, _onOnline = onOnline, _getIykcProofs = getIykcProofs, _setTimeout = setTimeout, _clearTimeout = clearTimeout, _random = Math.random, openDownloads = options => createChatOutbox({ ...options, namespace: 'downloads' }) }) {
|
|
23
35
|
fallbackRelays = normalizeFallbackRelays(fallbackRelays)
|
|
24
36
|
const reportedErrors = new WeakSet()
|
|
25
37
|
const reportError = error => {
|
|
@@ -40,6 +52,8 @@ export function createPrivateMessageSession ({ owner, signer, eventStore, messag
|
|
|
40
52
|
const sendControllers = new Map()
|
|
41
53
|
const offlineSends = new Set()
|
|
42
54
|
let stopOnline
|
|
55
|
+
let sendRetryTimer
|
|
56
|
+
let sendRetryDelay = SEND_RETRY_MIN_MS
|
|
43
57
|
let messenger
|
|
44
58
|
let fileTransfers
|
|
45
59
|
let downloads
|
|
@@ -60,6 +74,7 @@ export function createPrivateMessageSession ({ owner, signer, eventStore, messag
|
|
|
60
74
|
let lifecycle = 0
|
|
61
75
|
let configuring = Promise.resolve()
|
|
62
76
|
let sending
|
|
77
|
+
let repump
|
|
63
78
|
let draining
|
|
64
79
|
const idleWaiters = new Set()
|
|
65
80
|
const settled = () => { if (!sending && !draining) { for (const resolve of idleWaiters) resolve(); idleWaiters.clear() } }
|
|
@@ -179,9 +194,15 @@ export function createPrivateMessageSession ({ owner, signer, eventStore, messag
|
|
|
179
194
|
}
|
|
180
195
|
async function publish (peer, event, signal) {
|
|
181
196
|
if (deniedPeers.has(peer)) throw new Error('PERMISSION_DENIED')
|
|
182
|
-
if (!available || !peers.has(peer) || !channels.has(peer)) throw new Error('CHAT_UNAVAILABLE')
|
|
197
|
+
if (!available || !peers.has(peer) || !channels.has(peer)) throw unavailableError(new Error('CHAT_UNAVAILABLE'), signal)
|
|
183
198
|
const options = { channelPubkey: channels.get(peer)?.pubkey, receiverPubkeys: [peer], signal }
|
|
184
|
-
|
|
199
|
+
let report
|
|
200
|
+
try {
|
|
201
|
+
report = event.sig ? await messenger.broadcastEvent({ ...options, event }) : await messenger.broadcastRumor({ ...options, rumor: event })
|
|
202
|
+
} catch (error) {
|
|
203
|
+
if (isUnavailableSend(error)) throw unavailableError(error, signal)
|
|
204
|
+
throw error
|
|
205
|
+
}
|
|
185
206
|
// Published is relay acceptance, never a peer receipt.
|
|
186
207
|
await assertMessagePublished(report, event)
|
|
187
208
|
}
|
|
@@ -245,7 +266,10 @@ export function createPrivateMessageSession ({ owner, signer, eventStore, messag
|
|
|
245
266
|
await storage.remove(entry.id)
|
|
246
267
|
entries.delete(entry.id)
|
|
247
268
|
emit()
|
|
248
|
-
} catch (
|
|
269
|
+
} catch (cause) {
|
|
270
|
+
// Pauses can surface at any stage (text publication, file authorization
|
|
271
|
+
// or chunk delivery). Normalize all of them into the retryable path.
|
|
272
|
+
const error = cause?.retryWhenAvailable ? cause : (isUnavailableSend(cause) ? unavailableError(cause, controller.signal) : cause)
|
|
249
273
|
if (cancelled.has(entry.id)) { await storage.remove(entry.id); entries.delete(entry.id) } else {
|
|
250
274
|
entry.status = error.retryWhenAvailable ? 'pending' : 'error'; entry.failed = true; entry.retryable = retryable(error)
|
|
251
275
|
await storage.put(entry, { existing: true }).catch(reportError)
|
|
@@ -253,12 +277,42 @@ export function createPrivateMessageSession ({ owner, signer, eventStore, messag
|
|
|
253
277
|
// The failed wire event may be a quote or file chunk. Report the
|
|
254
278
|
// owning outbox item separately, without mutating the native error.
|
|
255
279
|
if (error.retryWhenOnline) offlineSends.add(entry.id)
|
|
280
|
+
if (error.retryWhenAvailable) scheduleSendRetry()
|
|
256
281
|
if (!closed && !entry.deletion && !cancelled.has(entry.id) && !error.retryWhenAvailable) onSendError(error, { id: entry.id, peer: entry.peer })
|
|
257
282
|
}
|
|
258
283
|
emit()
|
|
259
284
|
} finally { sendControllers.delete(entry.id) }
|
|
260
285
|
}
|
|
261
286
|
function stopOnlineRetry () { stopOnline?.(); stopOnline = undefined }
|
|
287
|
+
// Entries that failed only because the messenger was paused/unavailable do
|
|
288
|
+
// not receive a connectivity transition, so they own a bounded backoff timer
|
|
289
|
+
// instead of waiting for the next account-state change.
|
|
290
|
+
function clearSendRetry () {
|
|
291
|
+
if (sendRetryTimer) _clearTimeout(sendRetryTimer)
|
|
292
|
+
sendRetryTimer = undefined
|
|
293
|
+
}
|
|
294
|
+
function resetSendRetry () {
|
|
295
|
+
clearSendRetry()
|
|
296
|
+
sendRetryDelay = SEND_RETRY_MIN_MS
|
|
297
|
+
}
|
|
298
|
+
const pendingRetryable = entry => Boolean(entry.failed && !cancelled.has(entry.id) && (entry.retryable || entry.status === 'pending'))
|
|
299
|
+
function scheduleSendRetry () {
|
|
300
|
+
if (closed || !available || sendRetryTimer) return
|
|
301
|
+
const delay = Math.min(SEND_RETRY_MAX_MS, sendRetryDelay * (0.8 + _random() * 0.4))
|
|
302
|
+
sendRetryDelay = Math.min(SEND_RETRY_MAX_MS, sendRetryDelay * 2)
|
|
303
|
+
sendRetryTimer = _setTimeout(() => {
|
|
304
|
+
sendRetryTimer = undefined
|
|
305
|
+
if (closed || !available) return
|
|
306
|
+
let retried = false
|
|
307
|
+
for (const entry of entries.values()) {
|
|
308
|
+
if (!pendingRetryable(entry)) continue
|
|
309
|
+
entry.failed = false
|
|
310
|
+
retried = true
|
|
311
|
+
}
|
|
312
|
+
if (retried) pump()
|
|
313
|
+
}, delay)
|
|
314
|
+
sendRetryTimer.unref?.()
|
|
315
|
+
}
|
|
262
316
|
function watchOfflineSends () {
|
|
263
317
|
if (closed || !available || !offlineSends.size) { stopOnlineRetry(); return }
|
|
264
318
|
if (stopOnline) return
|
|
@@ -276,15 +330,26 @@ export function createPrivateMessageSession ({ owner, signer, eventStore, messag
|
|
|
276
330
|
})
|
|
277
331
|
}
|
|
278
332
|
async function pump () {
|
|
279
|
-
if (
|
|
333
|
+
if (!available || closed || !messenger || !storage) return
|
|
334
|
+
// A retry that lands while a pump is already running must not be skipped
|
|
335
|
+
// just because that pass already walked past the entry.
|
|
336
|
+
if (sending) { repump = true; return }
|
|
280
337
|
sending = true
|
|
281
338
|
try {
|
|
282
|
-
|
|
283
|
-
|
|
284
|
-
|
|
285
|
-
|
|
286
|
-
|
|
287
|
-
|
|
339
|
+
do {
|
|
340
|
+
repump = false
|
|
341
|
+
for (const entry of entries.values()) {
|
|
342
|
+
if (closed || !available) break
|
|
343
|
+
if ((entry.peer !== owner && !peers.has(entry.peer)) || entry.failed || cancelled.has(entry.id)) continue
|
|
344
|
+
await sendEntry(entry)
|
|
345
|
+
}
|
|
346
|
+
} while (repump && !closed && available)
|
|
347
|
+
} finally {
|
|
348
|
+
sending = false
|
|
349
|
+
watchOfflineSends()
|
|
350
|
+
if (![...entries.values()].some(pendingRetryable)) resetSendRetry()
|
|
351
|
+
settled()
|
|
352
|
+
}
|
|
288
353
|
}
|
|
289
354
|
const intentId = file => getEventHash({ kind: 0, pubkey: owner, created_at: 0, tags: [], content: `${file.peer}:${file.root}` })
|
|
290
355
|
async function download (file, options = {}) {
|
|
@@ -359,10 +424,12 @@ export function createPrivateMessageSession ({ owner, signer, eventStore, messag
|
|
|
359
424
|
async setAvailable (value) {
|
|
360
425
|
available = value === true
|
|
361
426
|
stopOnlineRetry(); offlineSends.clear()
|
|
427
|
+
if (available) resetSendRetry()
|
|
428
|
+
else clearSendRetry()
|
|
362
429
|
lifecycle++
|
|
363
430
|
if (!available) { await messenger?.pause('signer'); return }
|
|
364
431
|
await ready()
|
|
365
|
-
for (const entry of entries.values()) if (entry
|
|
432
|
+
for (const entry of entries.values()) if (pendingRetryable(entry)) entry.failed = false
|
|
366
433
|
await messenger?.resume('storage')
|
|
367
434
|
return schedule()
|
|
368
435
|
},
|
|
@@ -382,7 +449,7 @@ export function createPrivateMessageSession ({ owner, signer, eventStore, messag
|
|
|
382
449
|
offlineSends.delete(id); watchOfflineSends()
|
|
383
450
|
await ready()
|
|
384
451
|
const entry = entries.get(id)
|
|
385
|
-
if (entry) { entry.failed = false; if (deniedPeers.delete(entry.peer)) await schedule(); return pump() }
|
|
452
|
+
if (entry) { entry.failed = false; resetSendRetry(); if (deniedPeers.delete(entry.peer)) await schedule(); return pump() }
|
|
386
453
|
},
|
|
387
454
|
async cancel (id) {
|
|
388
455
|
cancelled.add(id)
|
|
@@ -396,6 +463,7 @@ export function createPrivateMessageSession ({ owner, signer, eventStore, messag
|
|
|
396
463
|
async close () {
|
|
397
464
|
closed = true; available = false; lifecycle++
|
|
398
465
|
stopOnlineRetry(); offlineSends.clear(); activeDownloads.clear(); pendingPriorityRanges.clear()
|
|
466
|
+
clearSendRetry()
|
|
399
467
|
for (const controller of sendControllers.values()) controller.abort()
|
|
400
468
|
await Promise.allSettled([configuring, initialized])
|
|
401
469
|
await messenger?.pause('closed')
|
|
@@ -0,0 +1,15 @@
|
|
|
1
|
+
const TRANSPORT_CATEGORIES = new Set(['connection', 'transport', 'timeout'])
|
|
2
|
+
const RETRY_PREFIXES = new Set(['rate-limited', 'error'])
|
|
3
|
+
const REPLACEMENT_PREFIXES = new Set(['blocked', 'restricted', 'auth-required', 'pow', ...RETRY_PREFIXES])
|
|
4
|
+
|
|
5
|
+
// Retry and replacement are different policies: a refusal can justify another
|
|
6
|
+
// relay without making the same operation retryable on the refusing relay.
|
|
7
|
+
function matches (error, prefixes) {
|
|
8
|
+
if (error?.name === 'Nip42AuthenticationError') return false
|
|
9
|
+
if (TRANSPORT_CATEGORIES.has(error?.category)) return true
|
|
10
|
+
if (error?.category && error.category !== 'relay') return false
|
|
11
|
+
return prefixes.has(/^([a-z-]+):/.exec(error?.message || '')?.[1])
|
|
12
|
+
}
|
|
13
|
+
|
|
14
|
+
export const isRetryableRelayFailure = error => matches(error, RETRY_PREFIXES)
|
|
15
|
+
export const isReplaceableRelayFailure = error => matches(error, REPLACEMENT_PREFIXES)
|
package/relay/index.js
CHANGED
|
@@ -1,4 +1,5 @@
|
|
|
1
1
|
export { freeRelays, nappRelays, seedRelays } from './constants/index.js'
|
|
2
|
+
export { isRetryableRelayFailure, isReplaceableRelayFailure } from './helpers/failure.js'
|
|
2
3
|
export { pickRelaysForPubkeys } from './helpers/routing.js'
|
|
3
4
|
export { RelayPool, relayPool } from './services/relay-pool.js'
|
|
4
5
|
export { getLatestEventsByPubkey } from './services/events.js'
|
package/relay/services/events.js
CHANGED
|
@@ -45,7 +45,7 @@ function getSelectedRelaysByPubkey (pubkeys, relayToAuthors) {
|
|
|
45
45
|
}
|
|
46
46
|
|
|
47
47
|
// Fetches batched events and keeps the newest event per address.
|
|
48
|
-
async function fetchLatestEventsByRelay (relayToAuthors, { kinds, dTagsByPubkey, getEvents }) {
|
|
48
|
+
async function fetchLatestEventsByRelay (relayToAuthors, { kinds, dTagsByPubkey, getEvents, report, phase, signal }) {
|
|
49
49
|
const requests = []
|
|
50
50
|
for (const [relay, authors] of relayToAuthors) {
|
|
51
51
|
const authorsByD = new Map()
|
|
@@ -57,8 +57,17 @@ async function fetchLatestEventsByRelay (relayToAuthors, { kinds, dTagsByPubkey,
|
|
|
57
57
|
for (const [d, dAuthors] of authorsByD) {
|
|
58
58
|
const filter = { kinds, authors: dAuthors }
|
|
59
59
|
if (d) filter['#d'] = [d]
|
|
60
|
-
requests.push(
|
|
61
|
-
|
|
60
|
+
requests.push(Promise.resolve().then(async () => {
|
|
61
|
+
signal?.throwIfAborted()
|
|
62
|
+
try {
|
|
63
|
+
const response = await getEvents(filter, [relay], signal ? { signal } : {})
|
|
64
|
+
report({ phase, authors: dAuthors, relays: response.relays || [], errors: response.errors || [] })
|
|
65
|
+
return { requested: new Set(dAuthors), events: (response.result || []).map(({ event }) => event) }
|
|
66
|
+
} catch (error) {
|
|
67
|
+
report({ phase, authors: dAuthors, relays: [{ relay, status: 'error', error }], error })
|
|
68
|
+
throw error
|
|
69
|
+
}
|
|
70
|
+
}))
|
|
62
71
|
}
|
|
63
72
|
}
|
|
64
73
|
|
|
@@ -95,11 +104,20 @@ export async function getLatestEventsByPubkey (pubkeys, {
|
|
|
95
104
|
fallbackRelays = freeRelays.slice(0, DEFAULT_FALLBACK_RELAY_COUNT),
|
|
96
105
|
relaysByPubkey,
|
|
97
106
|
relayListOptions,
|
|
107
|
+
excludeRelaysByPubkey,
|
|
108
|
+
signal,
|
|
109
|
+
onQueryResult,
|
|
98
110
|
_getRelaysByPubkey = getRelaysByPubkey,
|
|
99
111
|
_getEvents = getEvents
|
|
100
112
|
} = {}) {
|
|
113
|
+
signal?.throwIfAborted()
|
|
114
|
+
const requests = []
|
|
115
|
+
const report = request => { requests.push(request); onQueryResult?.(request) }
|
|
116
|
+
const excluded = pubkey => excludeRelaysByPubkey instanceof Map
|
|
117
|
+
? excludeRelaysByPubkey.get(pubkey) || []
|
|
118
|
+
: excludeRelaysByPubkey?.[pubkey] || []
|
|
101
119
|
const authors = [...new Set(pubkeys || [])].filter(Boolean)
|
|
102
|
-
if (!authors.length) return { events: [], byPubkey: {}, relaysByPubkey: {} }
|
|
120
|
+
if (!authors.length) return { events: [], byPubkey: {}, relaysByPubkey: {}, requests }
|
|
103
121
|
if (!Array.isArray(kinds) || kinds.length === 0) {
|
|
104
122
|
throw new ValidationError('MISSING_EVENT_KINDS', { message: 'Missing kinds' })
|
|
105
123
|
}
|
|
@@ -110,9 +128,16 @@ export async function getLatestEventsByPubkey (pubkeys, {
|
|
|
110
128
|
if (missingRelayAuthors.length) {
|
|
111
129
|
let discovered
|
|
112
130
|
try {
|
|
113
|
-
discovered = await _getRelaysByPubkey(missingRelayAuthors,
|
|
131
|
+
discovered = await _getRelaysByPubkey(missingRelayAuthors, {
|
|
132
|
+
...relayListOptions,
|
|
133
|
+
...(signal ? { signal } : {}),
|
|
134
|
+
onQueryResult: request => { report({ ...request, phase: 'discovery' }); relayListOptions?.onQueryResult?.(request) }
|
|
135
|
+
})
|
|
114
136
|
} catch (error) {
|
|
115
|
-
|
|
137
|
+
signal?.throwIfAborted()
|
|
138
|
+
if (!requests.some(request => request.phase === 'discovery' && request.error === error)) {
|
|
139
|
+
report({ phase: 'discovery', authors: missingRelayAuthors, relays: [], error })
|
|
140
|
+
}
|
|
116
141
|
discovered = Object.fromEntries(missingRelayAuthors.map(pubkey => [pubkey, { read: [], write: [] }]))
|
|
117
142
|
}
|
|
118
143
|
for (const pubkey of missingRelayAuthors) {
|
|
@@ -120,14 +145,17 @@ export async function getLatestEventsByPubkey (pubkeys, {
|
|
|
120
145
|
}
|
|
121
146
|
}
|
|
122
147
|
|
|
148
|
+
signal?.throwIfAborted()
|
|
123
149
|
const primaryAuthors = authors.filter(pubkey => relaysByAuthor[pubkey]?.[type]?.length)
|
|
124
|
-
const primaryRoutes = pickRelaysForPubkeys(primaryAuthors, relaysByAuthor, { maxPerPubkey, relayType })
|
|
150
|
+
const primaryRoutes = pickRelaysForPubkeys(primaryAuthors, relaysByAuthor, { maxPerPubkey, relayType, excludeRelaysByPubkey, emptyRelaysFallback: [] })
|
|
125
151
|
const selectedRelays = getSelectedRelaysByPubkey(authors, primaryRoutes)
|
|
152
|
+
for (const pubkey of authors) for (const relay of excluded(pubkey)) selectedRelays[pubkey].add(relay)
|
|
126
153
|
const latestByAddress = await fetchLatestEventsByRelay(primaryRoutes, {
|
|
127
154
|
kinds,
|
|
128
155
|
dTagsByPubkey,
|
|
129
|
-
getEvents: _getEvents
|
|
156
|
+
getEvents: _getEvents, report, phase: 'primary', signal
|
|
130
157
|
})
|
|
158
|
+
signal?.throwIfAborted()
|
|
131
159
|
const foundPubkeys = new Set(Object.values(latestByAddress).map(event => event.pubkey))
|
|
132
160
|
const missingAuthors = authors.filter(pubkey => !foundPubkeys.has(pubkey))
|
|
133
161
|
|
|
@@ -145,13 +173,14 @@ export async function getLatestEventsByPubkey (pubkeys, {
|
|
|
145
173
|
await fetchLatestEventsByRelay(fallbackRoutes, {
|
|
146
174
|
kinds,
|
|
147
175
|
dTagsByPubkey,
|
|
148
|
-
getEvents: _getEvents
|
|
176
|
+
getEvents: _getEvents, report, phase: 'fallback', signal
|
|
149
177
|
})
|
|
150
178
|
))
|
|
151
179
|
}
|
|
152
180
|
|
|
181
|
+
signal?.throwIfAborted()
|
|
153
182
|
const events = Object.values(latestByAddress)
|
|
154
183
|
const byPubkey = {}
|
|
155
184
|
for (const event of events) byPubkey[event.pubkey] = event
|
|
156
|
-
return { events, byPubkey, relaysByPubkey: relaysByAuthor }
|
|
185
|
+
return { events, byPubkey, relaysByPubkey: relaysByAuthor, requests }
|
|
157
186
|
}
|
package/relay/services/query.js
CHANGED
|
@@ -224,28 +224,27 @@ export function subscribeRelayListUpdates (pubkeys, {
|
|
|
224
224
|
}
|
|
225
225
|
|
|
226
226
|
async function loadMissingRelays (missingPubkeys, {
|
|
227
|
-
getEvents,
|
|
228
|
-
|
|
229
|
-
timeout = RELAY_LIST_QUERY_TIMEOUT_MS,
|
|
230
|
-
timeoutAfterFirstEose = RELAY_LIST_QUERY_TIMEOUT_AFTER_FIRST_EOSE_MS,
|
|
231
|
-
relayUrlPolicy,
|
|
232
|
-
emptyRelaysFallback = freeRelays.slice(0, 2)
|
|
227
|
+
getEvents, cacheMs, timeout, timeoutAfterFirstEose, relayUrlPolicy,
|
|
228
|
+
emptyRelaysFallback, excludeRelays, signal
|
|
233
229
|
}) {
|
|
234
|
-
|
|
235
|
-
|
|
236
|
-
authors: missingPubkeys,
|
|
237
|
-
|
|
238
|
-
|
|
239
|
-
|
|
240
|
-
|
|
241
|
-
}
|
|
242
|
-
|
|
230
|
+
let response
|
|
231
|
+
try {
|
|
232
|
+
response = await getEvents({ kinds: [10002], authors: missingPubkeys, limit: missingPubkeys.length },
|
|
233
|
+
seedRelays.filter(relay => !excludeRelays.includes(relay)), { timeout, timeoutAfterFirstEose, signal })
|
|
234
|
+
signal.throwIfAborted()
|
|
235
|
+
} catch (error) {
|
|
236
|
+
return { entries: {}, report: { authors: missingPubkeys, relays: [], error } }
|
|
237
|
+
}
|
|
238
|
+
const report = { authors: missingPubkeys, relays: response.relays || [], errors: response.errors || [] }
|
|
239
|
+
const complete = response.relays
|
|
240
|
+
? response.relays.length > 0 && response.relays.every(item => item.status === 'eose')
|
|
241
|
+
: !response.errors?.length
|
|
243
242
|
const latestByPubkey = {}
|
|
244
|
-
for (const { event } of
|
|
243
|
+
for (const { event } of response.result || []) {
|
|
245
244
|
if (!missingPubkeys.includes(event.pubkey)) continue
|
|
246
245
|
if (isNewerRelayListEvent(event, latestByPubkey[event.pubkey])) latestByPubkey[event.pubkey] = event
|
|
247
246
|
}
|
|
248
|
-
|
|
247
|
+
const entries = {}
|
|
249
248
|
for (const pubkey of missingPubkeys) {
|
|
250
249
|
const fetchedEvent = latestByPubkey[pubkey]
|
|
251
250
|
const cachedEvent = relayCacheEventByPubkey[pubkey] || null
|
|
@@ -253,9 +252,29 @@ async function loadMissingRelays (missingPubkeys, {
|
|
|
253
252
|
const relays = event
|
|
254
253
|
? parseRelayListEvent(event, relayUrlPolicy)
|
|
255
254
|
: { read: [...emptyRelaysFallback], write: [...emptyRelaysFallback] }
|
|
256
|
-
|
|
255
|
+
entries[pubkey] = { ...relays, event }
|
|
256
|
+
// A failed/incomplete lookup may use fallbacks for this call, but must not
|
|
257
|
+
// turn that failure into a forty-minute negative discovery cache entry.
|
|
258
|
+
if (fetchedEvent || complete) setCachedRelays(pubkey, relays, event, cacheMs)
|
|
257
259
|
}
|
|
258
260
|
pruneRelayCache()
|
|
261
|
+
return { entries, report }
|
|
262
|
+
}
|
|
263
|
+
|
|
264
|
+
// Each caller owns its wait, not another caller's shared discovery request.
|
|
265
|
+
function waitForDiscovery (record, signal) {
|
|
266
|
+
signal?.throwIfAborted()
|
|
267
|
+
record.users++
|
|
268
|
+
return new Promise((resolve, reject) => {
|
|
269
|
+
const abort = () => reject(signal.reason)
|
|
270
|
+
signal?.addEventListener('abort', abort, { once: true })
|
|
271
|
+
record.promise.then(resolve, reject).finally(() => signal?.removeEventListener('abort', abort))
|
|
272
|
+
}).finally(() => {
|
|
273
|
+
if (--record.users === 0 && !record.finished) {
|
|
274
|
+
record.controller.abort()
|
|
275
|
+
record.remove()
|
|
276
|
+
}
|
|
277
|
+
})
|
|
259
278
|
}
|
|
260
279
|
|
|
261
280
|
export async function getRelaysByPubkey (pubkeys, {
|
|
@@ -266,40 +285,48 @@ export async function getRelaysByPubkey (pubkeys, {
|
|
|
266
285
|
timeout = RELAY_LIST_QUERY_TIMEOUT_MS,
|
|
267
286
|
timeoutAfterFirstEose = RELAY_LIST_QUERY_TIMEOUT_AFTER_FIRST_EOSE_MS,
|
|
268
287
|
relayUrlPolicy,
|
|
269
|
-
emptyRelaysFallback = freeRelays.slice(0, 2)
|
|
288
|
+
emptyRelaysFallback = freeRelays.slice(0, 2),
|
|
289
|
+
excludeRelays = [],
|
|
290
|
+
signal,
|
|
291
|
+
onQueryResult
|
|
270
292
|
} = {}) {
|
|
293
|
+
signal?.throwIfAborted()
|
|
271
294
|
const pubkeyList = uniquePubkeys(pubkeys, { requireHex: _getEvents === getEvents })
|
|
272
295
|
if (!pubkeyList.length) return {}
|
|
273
|
-
|
|
274
|
-
const
|
|
275
|
-
|
|
276
|
-
|
|
277
|
-
const pubkeysToLoad = loadPubkeys.filter(pubkey => !relayRequestsByPubkey.has(pubkey))
|
|
296
|
+
// Different exclusions cannot share a request that contacts a refused relay.
|
|
297
|
+
const scope = JSON.stringify([...new Set(excludeRelays)].sort())
|
|
298
|
+
const key = pubkey => `${pubkey}:${scope}`
|
|
299
|
+
const loadPubkeys = forceRefresh ? pubkeyList : pubkeyList.filter(pubkey => !hasCachedKey(relaysByPubkey, pubkey))
|
|
300
|
+
const pubkeysToLoad = loadPubkeys.filter(pubkey => !relayRequestsByPubkey.has(key(pubkey)))
|
|
278
301
|
if (pubkeysToLoad.length) {
|
|
279
|
-
const
|
|
280
|
-
|
|
281
|
-
cacheMs,
|
|
282
|
-
timeout,
|
|
283
|
-
timeoutAfterFirstEose,
|
|
284
|
-
relayUrlPolicy,
|
|
285
|
-
emptyRelaysFallback
|
|
286
|
-
}).finally(() => {
|
|
302
|
+
const record = { controller: new AbortController(), users: 0, finished: false }
|
|
303
|
+
record.remove = () => {
|
|
287
304
|
for (const pubkey of pubkeysToLoad) {
|
|
288
|
-
if (relayRequestsByPubkey.get(pubkey) ===
|
|
305
|
+
if (relayRequestsByPubkey.get(key(pubkey)) === record) relayRequestsByPubkey.delete(key(pubkey))
|
|
289
306
|
}
|
|
290
|
-
}
|
|
291
|
-
|
|
307
|
+
}
|
|
308
|
+
record.promise = loadMissingRelays(pubkeysToLoad, {
|
|
309
|
+
getEvents: _getEvents, cacheMs, timeout, timeoutAfterFirstEose, relayUrlPolicy,
|
|
310
|
+
emptyRelaysFallback, excludeRelays, signal: record.controller.signal
|
|
311
|
+
}).finally(() => { record.finished = true; record.remove() })
|
|
312
|
+
for (const pubkey of pubkeysToLoad) relayRequestsByPubkey.set(key(pubkey), record)
|
|
292
313
|
}
|
|
293
|
-
|
|
294
|
-
|
|
295
|
-
|
|
296
|
-
|
|
297
|
-
|
|
298
|
-
|
|
299
|
-
|
|
300
|
-
.
|
|
301
|
-
|
|
302
|
-
|
|
303
|
-
|
|
304
|
-
|
|
314
|
+
const entries = {}
|
|
315
|
+
const records = [...new Set(loadPubkeys.map(pubkey => relayRequestsByPubkey.get(key(pubkey))).filter(Boolean))]
|
|
316
|
+
await Promise.all(records.map(async record => {
|
|
317
|
+
const result = await waitForDiscovery(record, signal)
|
|
318
|
+
signal?.throwIfAborted()
|
|
319
|
+
// Deliver the same native diagnostics to every interested caller, including
|
|
320
|
+
// callers that joined an already running query.
|
|
321
|
+
onQueryResult?.(result.report)
|
|
322
|
+
if (result.report.error) throw result.report.error
|
|
323
|
+
Object.assign(entries, result.entries)
|
|
324
|
+
}))
|
|
325
|
+
signal?.throwIfAborted()
|
|
326
|
+
return Object.fromEntries(pubkeyList.map(pubkey => {
|
|
327
|
+
const source = hasCachedKey(relaysByPubkey, pubkey) ? relaysByPubkey[pubkey] : entries[pubkey]
|
|
328
|
+
const entry = cloneRelays(source || { read: emptyRelaysFallback, write: emptyRelaysFallback })
|
|
329
|
+
if (includeEvents) entry.event = cloneRelayListEvent(relayCacheEventByPubkey[pubkey] || entries[pubkey]?.event)
|
|
330
|
+
return [pubkey, entry]
|
|
331
|
+
}))
|
|
305
332
|
}
|