libp2r2p 0.11.20 → 0.11.22

package/README.md

@@ -242,6 +242,49 @@ stream according to the existing partial-failure rules.
 
 ## Private Messenger
 
+### Pending ciphertext recovery (0.11.22)
+
+The known three-field message deduplication index is migrated automatically to
+include provenance, preserving positions, records and byte accounting. Other
+incompatible queue definitions still fail explicitly; no database reset is needed.
+
+PrivateMessenger defers `CONTENT_KEY_NOT_FOUND` and `INVALID_MAC` packets to its
+separate `${prefix}:pending-decryption:idb-queue`: 16 MiB FIFO, ciphertext only,
+no per-item TTL. Duplicate packets and attempts preserve their queue positions.
+Explicit channel removal and identity storage maintenance remove their pending
+records. Oversized records and capacity eviction are reported as discards.
+Stale channel metadata cleanup does not expire pending ciphertext.
+History continues only after deferral commits; storage/permission failures are
+not converted into successful coverage. Deferral is not message delivery.
+
+Call `retryPendingDecryptions()` after new content keys become available. Initial
+readiness and resume also request a pass. A single background worker coalesces
+requests, uses an origin/identity Web Lock when available, and stops on pause or
+close. Missing-key retries have no polling timer. MAC failures permit three total
+attempts, persisted across reloads; exhausted packets remain until FIFO eviction.
+`retryPendingDecryptions({ force: true })` explicitly retries exhausted packets.
+Success enters the normal acknowledged inbox before removing the pending item.
+The session coordinator forwards this API. Private-channel consumers may opt in
+with `onDecryptionFailure({ error, record })`, returning true only after durable
+handling, and replay records with `replayDeferredDecryptions`. Without the hook,
+standalone readers retain their existing error behavior.
+The optional `isDecryptionPending(record)` hook skips an already persisted
+packet before decryption; PrivateMessenger uses it for live/history/seeder
+duplicates so retransmission cannot bypass the MAC attempt budget.
+
+For standalone replay, pass `records: [{ id, channelPubkey, packet }]`, where
+`packet` is the ciphertext record received by the failure hook, together with
+the current receiver/channel signers and receiver pubkey. `onEvent` and
+`onNymEvent` admit normal deliveries; `onRecovered(record)` removes durable
+pending ciphertext only after admission, and `onFailure(record, error)` retains
+it and records the failed attempt. Both settlement callbacks are required.
+An optional AbortSignal fences processing; one batch shares a fragment processor.
+
+`rememberContentKeyEvent(event)` from `libp2r2p/content-key` validates and remembers
+a confirmed local announcement. Newest timestamp, then lowest ID, wins. Pending
+discovery queries cannot overwrite or return an older result; normal positive
+and negative cache windows remain unchanged.
+
 The main API is `createPrivateMessenger` from `libp2r2p/private-messenger`.
 It coordinates private-channel wrapping, relay watching, recovery state, and
 content-key lookup for direct or group-style private app messages.
@@ -692,6 +735,13 @@ Blossom server normalization accepts root `http:` origins for local
 development. Public Blossom validation additionally requires `https:` and a
 public host, and rejects credentials, paths, query strings and fragments.
 
+Starting with 0.11.21, import `CONTENT_KEY_ANNOUNCEMENT` (18716) from
+`libp2r2p/kind`. It is also included in `eventKinds` and used by content-key
+builders, proof validation and relay discovery. The announcement publishes a
+public content key for private communication and can be shared across apps.
+The former `CONTENT_KEY_KIND` export from `libp2r2p/content-key/event` has been
+removed; consumers must update that import.
+
 `classifyEvent()` from `libp2r2p/event` combines the exact NIP-01 kind
 ranges with tag-defined behavior. The first `d` tag may add `replaceable` or
 `addressable`, while an `expiration` tag equal to `created_at` adds

package/content-key/event/index.js

@@ -1,7 +1,6 @@
 import { ValidationError } from '../../error/index.js'
 import { getEventHash, isValidEvent } from '../../event/index.js'
-
-export const CONTENT_KEY_KIND = 18716
+import { CONTENT_KEY_ANNOUNCEMENT } from '../../kind/index.js'
 
 const HEX_PUBKEY = /^[0-9a-f]{64}$/
 const HEX_SIG = /^[0-9a-f]{128}$/i
@@ -15,7 +14,7 @@ export async function makeContentKeyEventForPubkey ({ userSigner, contentPubkey,
   if (!HEX_PUBKEY.test(contentPubkey || '')) throw new ValidationError('CONTENT_PUBKEY_REQUIRED')
 
   return userSigner.signEvent({
-    kind: CONTENT_KEY_KIND,
+    kind: CONTENT_KEY_ANNOUNCEMENT,
     created_at: createdAt,
     tags: [['cp', contentPubkey]],
     content: ''
@@ -32,7 +31,7 @@ export async function makeContentKeyEvent ({ userSigner, contentKeySigner, creat
 }
 
 export function parseContentKeyEvent (event) {
-  if (!event || event.kind !== CONTENT_KEY_KIND || event.content !== '') return null
+  if (!event || event.kind !== CONTENT_KEY_ANNOUNCEMENT || event.content !== '') return null
   if (!HEX_PUBKEY.test(event.pubkey) || !Number.isSafeInteger(event.created_at)) return null
   if (!Array.isArray(event.tags) || event.tags.length !== 1) return null
   if (!isValidEvent(event)) return null
@@ -67,7 +66,7 @@ function contentKeyProofError ({ ownerPubkey, contentPubkey, proof }) {
   if (!parsed) return 'INVALID_CONTENT_KEY_PROOF'
 
   const event = {
-    kind: CONTENT_KEY_KIND,
+    kind: CONTENT_KEY_ANNOUNCEMENT,
     pubkey: ownerPubkey,
     created_at: parsed.created_at,
     tags: [['cp', contentPubkey]],

package/content-key/index.js

@@ -1 +1 @@
-export { getIykcProofs } from './services/iykc-proof.js'
+export { getIykcProofs, rememberContentKeyEvent } from './services/iykc-proof.js'

package/content-key/services/iykc-proof.js

@@ -1,5 +1,7 @@
-import { CONTENT_KEY_KIND, parseContentKeyEvent } from '../event/index.js'
+import { parseContentKeyEvent } from '../event/index.js'
+import { CONTENT_KEY_ANNOUNCEMENT } from '../../kind/index.js'
 import { pickRelaysForPubkeys, getRelaysByPubkey, relayPool } from '../../relay/index.js'
+import { ValidationError } from '../../error/index.js'
 
 const QUERY_CACHE_MS = 40 * 60 * 1000
 const NEGATIVE_CACHE_MS = 5 * 60 * 1000
@@ -9,6 +11,7 @@ const contentKeysByPubkey = Object.create(null)
 const iykcCacheTimersByPubkey = Object.create(null)
 const iykcCacheAddedAtByPubkey = Object.create(null)
 const inFlightByPubkey = new Map()
+const announcementOrder = new Map()
 
 const getEvents = (...args) => relayPool.getEvents(...args)
 
@@ -36,6 +39,7 @@ function cloneContentKey (contentKey) {
 }
 
 function deleteCachedValue (cache, timers, addedAt, key) {
+  if (cache === contentKeysByPubkey) announcementOrder.delete(key)
   clearTimeout(timers[key])
   delete cache[key]
   delete timers[key]
@@ -71,6 +75,23 @@ export function clearContentKeyCache () {
   for (const key of Object.keys(iykcCacheTimersByPubkey)) delete iykcCacheTimersByPubkey[key]
   for (const key of Object.keys(iykcCacheAddedAtByPubkey)) delete iykcCacheAddedAtByPubkey[key]
   inFlightByPubkey.clear()
+  announcementOrder.clear()
+}
+
+// Remember only authenticated announcements, including those just published
+// locally. An older query may still finish, but cannot undo this selection.
+export function rememberContentKeyEvent (event) {
+  const proof = parseContentKeyEvent(event)
+  if (!proof) throw new ValidationError('INVALID_CONTENT_KEY_EVENT')
+  const previous = announcementOrder.get(event.pubkey)
+  if (previous && (previous.createdAt > event.created_at ||
+      (previous.createdAt === event.created_at && previous.id <= event.id))) return false
+  announcementOrder.set(event.pubkey, { createdAt: event.created_at, id: event.id })
+  setCachedValue(contentKeysByPubkey, iykcCacheTimersByPubkey, iykcCacheAddedAtByPubkey,
+    event.pubkey, cloneContentKey(proof), QUERY_CACHE_MS)
+  pruneCache(contentKeysByPubkey, iykcCacheTimersByPubkey, iykcCacheAddedAtByPubkey, IYKC_CACHE_MAX_ITEMS)
+  for (const key of announcementOrder.keys()) if (!hasCachedKey(contentKeysByPubkey, key)) announcementOrder.delete(key)
+  return true
 }
 
 async function queryContentKeyProofs (pubkeyList, { _getEvents, _getRelaysByPubkey, cacheMs, negativeCacheMs }) {
@@ -81,7 +102,7 @@ async function queryContentKeyProofs (pubkeyList, { _getEvents, _getRelaysByPubk
       .map(async ([relay, authors]) => {
         try {
           const { result } = await _getEvents({
-            kinds: [CONTENT_KEY_KIND],
+            kinds: [CONTENT_KEY_ANNOUNCEMENT],
             authors,
             limit: authors.length
           }, [relay], {
@@ -101,18 +122,28 @@ async function queryContentKeyProofs (pubkeyList, { _getEvents, _getRelaysByPubk
   for (const event of eventGroups.flat()) {
     const parsed = parseContentKeyEvent(event)
     if (!parsed) continue
-    if (!latestByPubkey[event.pubkey] || event.created_at > latestByPubkey[event.pubkey].created_at) {
-      latestByPubkey[event.pubkey] = { created_at: event.created_at, ...parsed }
+    const previous = latestByPubkey[event.pubkey]
+    if (!previous || event.created_at > previous.created_at ||
+        (event.created_at === previous.created_at && event.id < previous.id)) {
+      latestByPubkey[event.pubkey] = { created_at: event.created_at, id: event.id, ...parsed }
     }
   }
 
   const out = {}
   for (const pubkey of pubkeyList) {
     const entry = latestByPubkey[pubkey]
+    const selected = announcementOrder.get(pubkey)
+    if (selected && (!entry || selected.createdAt > entry.created_at ||
+        (selected.createdAt === entry.created_at && selected.id <= entry.id))) {
+      const cached = cloneContentKey(contentKeysByPubkey[pubkey])
+      if (cached) out[pubkey] = cached
+      continue
+    }
     const proof = entry
       ? { iykcPubkey: entry.iykcPubkey, iykcProof: entry.iykcProof }
       : null
     if (proof) {
+      announcementOrder.set(pubkey, { createdAt: entry.created_at, id: entry.id })
       setCachedValue(contentKeysByPubkey, iykcCacheTimersByPubkey, iykcCacheAddedAtByPubkey, pubkey, cloneContentKey(proof), cacheMs)
       out[pubkey] = proof
     } else if (negativeCacheMs > 0) {
@@ -122,6 +153,7 @@ async function queryContentKeyProofs (pubkeyList, { _getEvents, _getRelaysByPubk
     }
   }
   pruneCache(contentKeysByPubkey, iykcCacheTimersByPubkey, iykcCacheAddedAtByPubkey, IYKC_CACHE_MAX_ITEMS)
+  for (const key of announcementOrder.keys()) if (!hasCachedKey(contentKeysByPubkey, key)) announcementOrder.delete(key)
   return out
 }
 
@@ -168,5 +200,13 @@ export async function getIykcProofs (pubkeys, {
       if (cached) out[pubkey] = cached
     }
   }
+  // A batch may wait for another author after reading an already cached proof.
+  // Reconcile again before returning so a publication during that wait wins too.
+  for (const pubkey of pubkeyList) {
+    if (!hasCachedKey(contentKeysByPubkey, pubkey)) continue
+    const cached = cloneContentKey(contentKeysByPubkey[pubkey])
+    if (cached) out[pubkey] = cached
+    else delete out[pubkey]
+  }
   return out
 }

package/kind/index.js

@@ -58,6 +58,7 @@ export const FILE_SERVER_PREFERENCE = 10096
 export const GOOD_WIKI_AUTHORS = 10101
 export const GOOD_WIKI_RELAYS = 10102
 export const NWC_WALLET_INFO = 13194
+export const CONTENT_KEY_ANNOUNCEMENT = 18716
 export const LIGHTNING_PUB_RPC = 21000
 export const AUTH = 22242
 export const NWC_WALLET_REQUEST = 23194
@@ -198,6 +199,7 @@ export const eventKinds = /* @__PURE__ */ Object.freeze({
   GOOD_WIKI_AUTHORS,
   GOOD_WIKI_RELAYS,
   NWC_WALLET_INFO,
+  CONTENT_KEY_ANNOUNCEMENT,
   LIGHTNING_PUB_RPC,
   AUTH,
   NWC_WALLET_REQUEST,

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "libp2r2p",
-  "version": "0.11.20",
+  "version": "0.11.22",
   "description": "Peer-to-relay-to-peer",
   "keywords": [
     "p2r2p",

package/private-channel/helpers/decryption-error.js

@@ -0,0 +1,15 @@
+// Select only operationally useful public coordinates. Never attach a packet,
+// signer, plaintext, scope or message key to an error that can reach a logger.
+export function decryptionError (cause, context) {
+  const error = new Error(cause.message, { cause })
+  error.name = cause.name
+  const code = cause.code || (cause.message === 'CONTENT_KEY_NOT_FOUND' ? 'CONTENT_KEY_NOT_FOUND' : cause.message === 'invalid MAC' ? 'INVALID_MAC' : undefined)
+  if (code) error.code = code
+  for (const key of ['stage', 'channelPubkey', 'eventId', 'eventKind', 'senderPubkey', 'receiverPubkey', 'ownContentPubkey', 'peerContentPubkey', 'hasOwnContentKey']) {
+    const value = context?.[key] ?? cause[key]
+    if (['string', 'number', 'boolean'].includes(typeof value)) error[key] = value
+  }
+  return error
+}
+
+export const isDeferredDecryptionError = error => ['CONTENT_KEY_NOT_FOUND', 'INVALID_MAC'].includes(error?.code)

package/private-channel/index.js

@@ -5,6 +5,7 @@ import { hexToBytes } from '../base16/index.js'
 import { isValidContentKeyProof, isValidIykcProof, makeContentKeyEventForPubkey, parseContentKeyEvent } from '../content-key/event/index.js'
 import { getIykcProofs } from '../content-key/index.js'
 import { ValidationError } from '../error/index.js'
+import { decryptionError, isDeferredDecryptionError } from './helpers/decryption-error.js'
 import { normalizeRumor, deliveryInfo } from './helpers/rumor.js'
 import { incompleteFetchError } from './helpers/fetch-error.js'
 import { readHistory } from './helpers/history.js'
@@ -416,19 +417,27 @@ async function unwrapRecipientEnvelope ({ payloadCiphertext, envelope, receiverS
     // no iykc pubkey, so hand imkc to the signer as the receiver's own key.
     const ownContentPubkey = envelope.iykcPubkey ||
       (senderPubkey && receiverPubkey && senderPubkey === receiverPubkey ? imkcPubkey : '')
-    messageSeckey = base64ToText(await receiverSigner.nip44DecryptDoubleDH(
-      senderPubkey,
-      ROUTER_KIND,
-      rowScope,
-      envelope.ciphertext,
-      imkcPubkey,
-      ownContentPubkey
-    ))
+    try {
+      messageSeckey = base64ToText(await receiverSigner.nip44DecryptDoubleDH(
+        senderPubkey,
+        ROUTER_KIND,
+        rowScope,
+        envelope.ciphertext,
+        imkcPubkey,
+        ownContentPubkey
+      ))
+    } catch (cause) {
+      throw decryptionError(cause, { stage: 'recipient-envelope', channelPubkey: rowScope, senderPubkey, receiverPubkey, ownContentPubkey, peerContentPubkey: imkcPubkey })
+    }
   } else {
     if (!receiverSigner?.nip44v3Decrypt) throw new ValidationError('RECEIVER_SIGNER_NIP44V3_DECRYPT_UNSUPPORTED')
-    messageSeckey = base64ToText(await receiverSigner.nip44v3Decrypt(senderPubkey, ROUTER_KIND, rowScope, envelope.ciphertext))
+    try {
+      messageSeckey = base64ToText(await receiverSigner.nip44v3Decrypt(senderPubkey, ROUTER_KIND, rowScope, envelope.ciphertext))
+    } catch (cause) { throw decryptionError(cause, { stage: 'recipient-envelope', channelPubkey: rowScope, senderPubkey, receiverPubkey }) }
+  }
+  try { return eventFromPayload({ payloadCiphertext, messageSeckey, senderPubkey }) } catch (cause) {
+    throw decryptionError(cause, { stage: 'message-payload', channelPubkey: rowScope, senderPubkey, receiverPubkey })
   }
-  return eventFromPayload({ payloadCiphertext, messageSeckey, senderPubkey })
 }
 
 export async function unwrapEvent ({ receiverSigner, privateChannelSigner = receiverSigner, privateChannelReaderSigner = privateChannelSigner, privateChannelReaderPubkey, event, receiverPubkey }) {
@@ -708,6 +717,7 @@ function createProcessor ({
   onNymEvent,
   onSeedEvent,
   onContentKeyUsage,
+  onDecryptionFailure, isDecryptionPending,
   onError,
   receivedChunkScope = '',
   receivedChunkTtlMs = DEFAULT_RECEIVED_CHUNK_TTL_MS,
@@ -733,6 +743,7 @@ function createProcessor ({
   async function processOuterEvent (outer) {
     await storeReady
     let groupKey = ''
+    let routerDecrypted = false
     try {
       const channelPubkey = outer.pubkey || await privateChannelSigner?.getPublicKey?.()
       const channelSigner = readSignerFromMap(privateChannelSignersByPubkey, channelPubkey) || privateChannelSigner
@@ -742,6 +753,7 @@ function createProcessor ({
       const channelReceivedChunkTtlMs = readValueFromMap(receivedChunkTtlMsByPubkey, channelPubkey) ?? receivedChunkTtlMs
       if (!channelPubkey) throw new ValidationError('PRIVATE_CHANNEL_PUBKEY_REQUIRED')
 
+      if (await isDecryptionPending?.({ format: 'outer_v1', channelPubkey, outer })) return
       const decrypted = await decryptRouter({
         content: outer.content,
         channelPubkey,
@@ -749,6 +761,7 @@ function createProcessor ({
         channelReaderSigner,
         channelReaderPubkey
       })
+      routerDecrypted = true
       if (decrypted.kind === NYM_CARRIER_KIND) {
         const carrier = assertValidNymCarrierEvent(decrypted)
         const { index, total } = readChunkTag(carrier)
@@ -834,6 +847,7 @@ function createProcessor ({
       // regular leechers can stop as soon as their envelope is decrypted.
       const mustScanWholeBundle = shouldSeed || sentByReceiver
       let event = null
+      let deferred = false
       const innerEventIdsByRowIndex = {}
 
       const drained = await receivedChunks.drainAvailable(groupKey, {
@@ -858,15 +872,36 @@ function createProcessor ({
               onContentKeyUsage
             })
             if (receiverSigner && !event) {
-              event = await unwrapRecipientEnvelope({
-                payloadCiphertext: groupMeta.payloadCiphertext,
-                envelope,
-                receiverSigner,
-                receiverPubkey,
-                senderPubkey,
-                imkcPubkey,
-                rowScope: channelPubkey
-              })
+              const record = () => {
+                const row = JSON.stringify([envelope.receiverPubkey, envelope.ciphertext, envelope.iykcPubkey, envelope.iykcProof])
+                return {
+                  format: 'routerRow_v1', channelPubkey,
+                  outer: { id: outer.id, kind: outer.kind, pubkey: outer.pubkey, created_at: outer.created_at },
+                  router: joinedRouter(router, bytesToBase64(encoder.encode(JSON.stringify([groupMeta.payloadCiphertext]) + '\n' + row + '\n')))
+                }
+              }
+              if (await isDecryptionPending?.(record())) {
+                deferred = true
+                if (!mustScanWholeBundle) return { stop: true }
+                return
+              }
+              try {
+                event = await unwrapRecipientEnvelope({
+                  payloadCiphertext: groupMeta.payloadCiphertext,
+                  envelope,
+                  receiverSigner,
+                  receiverPubkey,
+                  senderPubkey,
+                  imkcPubkey,
+                  rowScope: channelPubkey
+                })
+              } catch (cause) {
+                const error = decryptionError(cause, { eventId: outer.id, eventKind: outer.kind, channelPubkey })
+                if (!onDecryptionFailure || !isDeferredDecryptionError(error)) throw error
+                if (!await onDecryptionFailure({ error, record: record() })) throw error
+                deferred = true
+                if (!mustScanWholeBundle) return { stop: true }
+              }
               if (event) {
                 innerEventIdsByRowIndex[rowIndex] = event.id
                 if (!mustScanWholeBundle) return { stop: true }
@@ -876,8 +911,8 @@ function createProcessor ({
         }
       })
 
-      if (event && !mustScanWholeBundle) {
-        await onEvent?.(event, outer, { router: joinedRouter(router), channelPubkey, ...deliveryInfo(event, senderPubkey) })
+      if ((event || deferred) && !mustScanWholeBundle) {
+        if (event) await onEvent?.(event, outer, { router: joinedRouter(router), channelPubkey, ...deliveryInfo(event, senderPubkey) })
         ignoredGroups.add(groupKey)
         await receivedChunks.removeGroup(groupKey)
         return
@@ -902,7 +937,13 @@ function createProcessor ({
       if (event) await onEvent?.(event, outer, { router: completeRouter, channelPubkey, jsonl, ...deliveryInfo(event, senderPubkey) })
 
       await receivedChunks.removeGroup(groupKey)
-    } catch (err) {
+    } catch (cause) {
+      let err = cause
+      if (!routerDecrypted) {
+        err = decryptionError(err, { stage: 'outer-envelope', channelPubkey: outer.pubkey, eventId: outer.id, eventKind: outer.kind })
+        if (onDecryptionFailure && isDeferredDecryptionError(err) &&
+            await onDecryptionFailure({ error: err, record: { format: 'outer_v1', channelPubkey: outer.pubkey, outer } })) return
+      }
       if (shouldIgnoreGroupError(err) && groupKey) {
         ignoredGroups.add(groupKey)
         await receivedChunks.removeGroup(groupKey).catch(() => {})
@@ -939,7 +980,81 @@ function shouldIgnoreGroupError (err) {
   ].includes(err?.message)
 }
 
-export async function fetch ({ signal, receiverSigner, iykcSigner, privateChannelSigner = receiverSigner, privateChannelSignersByPubkey, privateChannelReaderSigner = privateChannelSigner, privateChannelReaderSignersByPubkey, privateChannelReaderPubkey, privateChannelReaderPubkeysByPubkey, privateChannelPubkey, privateChannelPubkeys, receiverPubkey, relays, onChunk, onEvent, onNymEvent, onSeedEvent, onContentKeyUsage, onError, since, until, limit, mode = 'leecher', modeByPubkey, receivedChunkScope = '', receivedChunkTtlMs = DEFAULT_RECEIVED_CHUNK_TTL_MS, receivedChunkTtlMsByPubkey, receivedChunkMaxBytes = DEFAULT_RECEIVED_CHUNK_MAX_BYTES, receivedChunkIndexedDB = globalThis.indexedDB, ignoredGroupTtlMs = DEFAULT_IGNORED_GROUP_TTL_MS, ignoredGroupMaxEntries = DEFAULT_IGNORED_GROUP_MAX_ENTRIES, _getEvents = getEvents }) {
+// Replay ciphertext only. Keep one fragment processor for the entire batch;
+// incomplete outer groups remain pending until a later key/channel transition.
+export async function replayDeferredDecryptions ({ records, onRecovered, onFailure, ...options }) {
+  if (!Array.isArray(records) || records.some(record => typeof record?.id !== 'string' ||
+      typeof record.channelPubkey !== 'string' || !['outer_v1', 'routerRow_v1'].includes(record.packet?.format))) throw new ValidationError('INVALID_DEFERRED_RECORDS')
+  if (typeof onRecovered !== 'function' || typeof onFailure !== 'function') throw new ValidationError('DEFERRED_CALLBACKS_REQUIRED')
+  const groups = new Map()
+  const completed = new Set()
+  let current
+  let chunked = false
+  const processor = createProcessor({
+    ...options, onDecryptionFailure: undefined, isDecryptionPending: undefined,
+    onError: error => { throw error },
+    onChunk: chunk => {
+      chunked = true
+      const key = chunk.router?.pubkey || nymCarrierGroupId(chunk.nymCarrier)
+      if (!groups.has(key)) groups.set(key, new Set())
+      groups.get(key).add(current.id)
+    },
+    onEvent: async (event, outer, meta) => {
+      options.signal?.throwIfAborted()
+      await options.onEvent?.(event, outer, meta)
+      completed.add(meta.router?.pubkey)
+    },
+    onNymEvent: async (event, outer, meta) => {
+      options.signal?.throwIfAborted()
+      await options.onNymEvent?.(event, outer, meta)
+      completed.add(nymCarrierGroupId(meta.carrier))
+    },
+    onSeedEvent: async seed => {
+      options.signal?.throwIfAborted()
+      await options.onSeedEvent?.(seed)
+      completed.add(seed.router?.pubkey || nymCarrierGroupId(seed.carrier))
+    }
+  })
+  try {
+    for (const record of records) {
+      options.signal?.throwIfAborted()
+      current = record
+      chunked = false
+      try {
+        if (record.packet.format === 'outer_v1') {
+          await processor(record.packet.outer)
+          if (!chunked) await onRecovered(record)
+        } else {
+          const event = await unwrapRouterEvent({
+            receiverSigner: options.receiverSigner, receiverPubkey: options.receiverPubkey,
+            channelPubkey: record.channelPubkey, router: record.packet.router
+          })
+          options.signal?.throwIfAborted()
+          if (event) {
+            await options.onEvent?.(event, record.packet.outer, {
+              router: record.packet.router, channelPubkey: record.channelPubkey,
+              ...deliveryInfo(event, readSenderTag(record.packet.router))
+            })
+          }
+          await onRecovered(record)
+        }
+      } catch (error) {
+        options.signal?.throwIfAborted()
+        await onFailure(record, decryptionError(error, { channelPubkey: record.channelPubkey, eventId: record.packet.outer?.id }))
+      }
+      // Let live ingestion and foreground signer work run between records.
+      await new Promise(resolve => setTimeout(resolve, 0))
+    }
+    for (const key of completed) {
+      for (const id of groups.get(key) || []) {
+        const record = records.find(record => record.id === id)
+        if (record) await onRecovered(record)
+      }
+    }
+  } finally { processor.close() }
+}
+
+export async function fetch ({ signal, receiverSigner, iykcSigner, privateChannelSigner = receiverSigner, privateChannelSignersByPubkey, privateChannelReaderSigner = privateChannelSigner, privateChannelReaderSignersByPubkey, privateChannelReaderPubkey, privateChannelReaderPubkeysByPubkey, privateChannelPubkey, privateChannelPubkeys, receiverPubkey, relays, onChunk, onEvent, onNymEvent, onSeedEvent, onContentKeyUsage, onDecryptionFailure, isDecryptionPending, onError, since, until, limit, mode = 'leecher', modeByPubkey, receivedChunkScope = '', receivedChunkTtlMs = DEFAULT_RECEIVED_CHUNK_TTL_MS, receivedChunkTtlMsByPubkey, receivedChunkMaxBytes = DEFAULT_RECEIVED_CHUNK_MAX_BYTES, receivedChunkIndexedDB = globalThis.indexedDB, ignoredGroupTtlMs = DEFAULT_IGNORED_GROUP_TTL_MS, ignoredGroupMaxEntries = DEFAULT_IGNORED_GROUP_MAX_ENTRIES, _getEvents = getEvents }) {
   if (!relays?.length) throw new ValidationError('NO_RELAYS')
   const authors = privateChannelPubkeyList({ privateChannelPubkey, privateChannelPubkeys })
   const filter = { kinds: [PRIVATE_BROADCAST_KIND] }
@@ -958,7 +1073,7 @@ export async function fetch ({ signal, receiverSigner, iykcSigner, privateChanne
   const elapsedMs = Math.round(performance.now() - startedAt)
   const events = result.map(({ event }) => event)
   events.sort((a, b) => a.created_at - b.created_at)
-  const processOuterEvent = createProcessor({ receiverSigner, iykcSigner, privateChannelSigner, privateChannelSignersByPubkey, privateChannelReaderSigner, privateChannelReaderSignersByPubkey, privateChannelReaderPubkey, privateChannelReaderPubkeysByPubkey, receiverPubkey, mode, modeByPubkey, onChunk, onEvent, onNymEvent, onSeedEvent, onContentKeyUsage, onError, receivedChunkScope, receivedChunkTtlMs, receivedChunkTtlMsByPubkey, receivedChunkMaxBytes, receivedChunkIndexedDB, ignoredGroupTtlMs, ignoredGroupMaxEntries })
+  const processOuterEvent = createProcessor({ receiverSigner, iykcSigner, privateChannelSigner, privateChannelSignersByPubkey, privateChannelReaderSigner, privateChannelReaderSignersByPubkey, privateChannelReaderPubkey, privateChannelReaderPubkeysByPubkey, receiverPubkey, mode, modeByPubkey, onChunk, onEvent, onNymEvent, onSeedEvent, onContentKeyUsage, onDecryptionFailure, isDecryptionPending, onError, receivedChunkScope, receivedChunkTtlMs, receivedChunkTtlMsByPubkey, receivedChunkMaxBytes, receivedChunkIndexedDB, ignoredGroupTtlMs, ignoredGroupMaxEntries })
   try {
     for (const event of events) {
       signal?.throwIfAborted()
@@ -988,7 +1103,7 @@ export async function fetchHistory ({ receiverSigner, privateChannelSigner = rec
   } finally { processOuterEvent.close() }
 }
 
-export function subscribe ({ receiverSigner, iykcSigner, privateChannelSigner = receiverSigner, privateChannelSignersByPubkey, privateChannelReaderSigner = privateChannelSigner, privateChannelReaderSignersByPubkey, privateChannelReaderPubkey, privateChannelReaderPubkeysByPubkey, privateChannelPubkey, privateChannelPubkeys, receiverPubkey, relays, onChunk, onEvent, onNymEvent, onSeedEvent, onContentKeyUsage, onError, since = nowSeconds() - 5, limit, liveOnly = false, mode = 'leecher', modeByPubkey, receivedChunkScope = '', receivedChunkTtlMs = DEFAULT_RECEIVED_CHUNK_TTL_MS, receivedChunkTtlMsByPubkey, receivedChunkMaxBytes = DEFAULT_RECEIVED_CHUNK_MAX_BYTES, receivedChunkIndexedDB = globalThis.indexedDB, ignoredGroupTtlMs = DEFAULT_IGNORED_GROUP_TTL_MS, ignoredGroupMaxEntries = DEFAULT_IGNORED_GROUP_MAX_ENTRIES, _liveEventsGenerator = getLiveEventsGenerator, _eventsFeedGenerator = getEventsFeedGenerator }) {
+export function subscribe ({ receiverSigner, iykcSigner, privateChannelSigner = receiverSigner, privateChannelSignersByPubkey, privateChannelReaderSigner = privateChannelSigner, privateChannelReaderSignersByPubkey, privateChannelReaderPubkey, privateChannelReaderPubkeysByPubkey, privateChannelPubkey, privateChannelPubkeys, receiverPubkey, relays, onChunk, onEvent, onNymEvent, onSeedEvent, onContentKeyUsage, onDecryptionFailure, isDecryptionPending, onError, since = nowSeconds() - 5, limit, liveOnly = false, mode = 'leecher', modeByPubkey, receivedChunkScope = '', receivedChunkTtlMs = DEFAULT_RECEIVED_CHUNK_TTL_MS, receivedChunkTtlMsByPubkey, receivedChunkMaxBytes = DEFAULT_RECEIVED_CHUNK_MAX_BYTES, receivedChunkIndexedDB = globalThis.indexedDB, ignoredGroupTtlMs = DEFAULT_IGNORED_GROUP_TTL_MS, ignoredGroupMaxEntries = DEFAULT_IGNORED_GROUP_MAX_ENTRIES, _liveEventsGenerator = getLiveEventsGenerator, _eventsFeedGenerator = getEventsFeedGenerator }) {
   if (!relays?.length) throw new ValidationError('NO_RELAYS')
   if (receiverSigner && !receiverSigner?.nip44DecryptDoubleDH && !receiverSigner?.nip44v3Decrypt) throw new ValidationError('RECEIVER_SIGNER_NIP44V3_DECRYPT_UNSUPPORTED')
   if (!privateChannelReaderSigner && !privateChannelReaderSignersByPubkey && !privateChannelSigner && !privateChannelSignersByPubkey) throw new ValidationError('PRIVATE_CHANNEL_READER_REQUIRED')
@@ -997,7 +1112,7 @@ export function subscribe ({ receiverSigner, iykcSigner, privateChannelSigner =
   const filter = { kinds: [PRIVATE_BROADCAST_KIND], since }
   if (authors.length) filter.authors = authors
   if (limit != null) filter.limit = limit
-  const processOuterEvent = createProcessor({ receiverSigner, iykcSigner, privateChannelSigner, privateChannelSignersByPubkey, privateChannelReaderSigner, privateChannelReaderSignersByPubkey, privateChannelReaderPubkey, privateChannelReaderPubkeysByPubkey, receiverPubkey, mode, modeByPubkey, onChunk, onEvent, onNymEvent, onSeedEvent, onContentKeyUsage, onError, receivedChunkScope, receivedChunkTtlMs, receivedChunkTtlMsByPubkey, receivedChunkMaxBytes, receivedChunkIndexedDB, ignoredGroupTtlMs, ignoredGroupMaxEntries })
+  const processOuterEvent = createProcessor({ receiverSigner, iykcSigner, privateChannelSigner, privateChannelSignersByPubkey, privateChannelReaderSigner, privateChannelReaderSignersByPubkey, privateChannelReaderPubkey, privateChannelReaderPubkeysByPubkey, receiverPubkey, mode, modeByPubkey, onChunk, onEvent, onNymEvent, onSeedEvent, onContentKeyUsage, onDecryptionFailure, isDecryptionPending, onError, receivedChunkScope, receivedChunkTtlMs, receivedChunkTtlMsByPubkey, receivedChunkMaxBytes, receivedChunkIndexedDB, ignoredGroupTtlMs, ignoredGroupMaxEntries })
   const controller = new AbortController()
   const events = liveOnly
     ? _liveEventsGenerator(filter, relays, {

package/private-message/index.js

@@ -477,6 +477,8 @@ export function createPrivateMessageSession ({ _setTimeout = setTimeout, _clearT
           return dispatchSeedEvent(seed)
         },
         onContentKeyUsage: dispatchContentKeyUsage,
+        isDecryptionPending: record => watchCallbacks(record.channelPubkey).isDecryptionPending?.(record),
+        onDecryptionFailure: failure => watchCallbacks(failure.record.channelPubkey).onDecryptionFailure?.(failure),
         onError: err => reportSubscriptionError(channels, err)
       })
 
@@ -522,6 +524,7 @@ export function createPrivateMessageSession ({ _setTimeout = setTimeout, _clearT
     onSeed,
     onChunk,
     onContentKeyUsage,
+    onDecryptionFailure, isDecryptionPending,
     onSubscriptionState,
     onError,
     receivedChunkTtlMs,
@@ -542,7 +545,7 @@ export function createPrivateMessageSession ({ _setTimeout = setTimeout, _clearT
     const ownPubkey = receiverPubkey || await receiverSigner?.getPublicKey?.()
     if (identity !== undefined && identity !== ownPubkey) throw new ValidationError('PRIVATE_MESSAGE_IDENTITY_MISMATCH')
     identity = ownPubkey
-    const callbacks = { onAsk, onReply, onTell, onYell, onNym, onMessage, onSeed, onChunk, onContentKeyUsage, onSubscriptionState, onError }
+    const callbacks = { onAsk, onReply, onTell, onYell, onNym, onMessage, onSeed, onChunk, onContentKeyUsage, onDecryptionFailure, isDecryptionPending, onSubscriptionState, onError }
 
     for (const channel of channelList) {
       if (watchRequests.get(channel) !== request) continue

package/private-messenger/index.js

@@ -54,6 +54,9 @@ import { incompleteFetchError } from '../private-channel/helpers/fetch-error.js'
 import { recoveryRetryDelay, isPermanentRecoveryError } from './helpers/recovery-retry.js'
 import { normalizeRelayUrl } from '../url/index.js'
 import { deliveryInfo } from '../private-channel/helpers/rumor.js'
+import { migrateMessageQueue } from './services/migrate-message-queue.js'
+import { createPendingDecryption } from './services/pending-decryption.js'
+import { decryptionError, isDeferredDecryptionError } from '../private-channel/helpers/decryption-error.js'
 import { bytesToBase64 } from '../base64/index.js'
 import { ValidationError } from '../error/index.js'
 import { getRelaysByPubkey, pickRelaysForPubkeys, subscribeRelayListUpdates, relayPool } from '../relay/index.js'
@@ -419,6 +422,7 @@ export class PrivateMessenger {
       this.assertOpen()
       this.contentKeyPubkey = await this.contentKeySigner?.getPublicKey?.() || ''
       this.assertOpen()
+      await migrateMessageQueue(this._indexedDB, this.prefix)
       this.queue = await createQueue({
         prefix: this.prefix,
         indexes: MESSAGE_QUEUE_INDEXES,
@@ -427,6 +431,10 @@ export class PrivateMessenger {
         indexedDB: this._indexedDB
       })
       this.assertOpen()
+      this.pendingDecryption = await createPendingDecryption({
+        prefix: this.prefix, indexedDB: this._indexedDB,
+        report: detail => this.debug('pending-decryption', detail)
+      })
       this.seedQueue = this.seedStorage
         ? null
         : await createQueue({
@@ -446,16 +454,20 @@ export class PrivateMessenger {
       this.state = { channels: await this.stateStore.load() }
       await this.update({ userSigner, contentKeySigner, nymSigner: this.nymSigner, channels, relays, mode })
       this.initialized = true
+      this.retryPendingDecryptions().catch(error => this.onError?.(error))
       await this.refreshAndApplyStoragePolicy()
       this.broadcastStoragePolicyChange()
       return this
     } catch (err) {
+      this.decryptionRetry?.controller.abort(err)
+      await Promise.allSettled([this.decryptionRetry?.work])
       this.stopStorageMaintenance()
       this.stopStoragePolicyBroadcast()
       try { await this.queueOperationTail } catch {}
       try { await this.stateWriteTail } catch {}
       await Promise.allSettled([
         this.queue?.close?.(),
+        this.pendingDecryption?.close(),
         this.seedQueue?.close?.(),
         this.stateStore?.close?.()
       ])
@@ -1209,6 +1221,8 @@ export class PrivateMessenger {
           onMessage: message => this.receive(pubkey, () => this.handleMessage(pubkey, message), message),
           onSeed: seed => this.receive(pubkey, () => this.enqueueSeed(pubkey, seed), seed),
           onContentKeyUsage: usage => this.handleContentKeyUsage(pubkey, usage),
+          onDecryptionFailure: failure => this.deferDecryption(failure),
+          isDecryptionPending: record => this.pendingDecryption.has(record),
           receivedChunkTtlMs: this.receivedChunkTtlMsFor(channel),
           receivedChunkIndexedDB: this._indexedDB,
           onSubscriptionState: state => this.handleSubscriptionState(pubkey, state),
@@ -1241,6 +1255,11 @@ export class PrivateMessenger {
         continue
       }
       this.stopByChannel.set(pubkey, stop)
+      this.decryptionReadyChannels ??= new Set()
+      if (!this.decryptionReadyChannels.has(pubkey)) {
+        this.decryptionReadyChannels.add(pubkey)
+        this.retryPendingDecryptions().catch(error => this.onError?.(error))
+      }
       this.watchRevisionByChannel.set(pubkey, revision)
       this.updateChannelState(pubkey, {
         lastWatchedAt: nowSeconds(),
@@ -1829,6 +1848,7 @@ export class PrivateMessenger {
     if (typeof reason !== 'string' || !reason.trim()) throw new ValidationError('PAUSE_REASON_REQUIRED')
     this.assertOpen()
     this.pauseReasons.add(reason)
+    this.decryptionRetry?.controller.abort(this.pausedError())
     this.pauseRevisions.set(reason, (this.pauseRevisions.get(reason) || 0) + 1)
     this.sendRoutingPause.abort(this.pausedError())
     this.notifyStatus()
@@ -1865,6 +1885,7 @@ export class PrivateMessenger {
     }
     if (this.pauseReasons.size || this.closePromise) return
     const jobs = removed ? this.startChannelBackground(channels, { recover: true }) : [...this.channelBackground.values()].map(job => job.work)
+    if (removed) this.retryPendingDecryptions().catch(error => this.onError?.(error))
     if (removed) {
       for (const extension of this.extensions) {
         const work = Promise.resolve().then(() => { if (!this.pauseReasons.size && !this.closePromise) return extension.resume?.() })
@@ -2739,7 +2760,19 @@ export class PrivateMessenger {
     for (const line of splitJsonl(jsonl)) {
       const record = parseJson(line, null)
       if (!record) continue
-      const recovered = await this.messageFromBackfillRecord(channelPubkey, record)
+      const pendingRecord = record.recordType === ROUTER_SEED_RECORD_TYPE && isPrivateChannelRouter(record.router)
+        ? { format: 'routerRow_v1', channelPubkey, router: record.router, outer: { id: '', created_at: record.lastSeenAt || record.router.created_at } }
+        : null
+      if (pendingRecord && await this.pendingDecryption.has(pendingRecord)) continue
+      let recovered
+      try { recovered = await this.messageFromBackfillRecord(channelPubkey, record) } catch (cause) {
+        const error = decryptionError(cause, { channelPubkey })
+        if (!isDeferredDecryptionError(error) || record.recordType !== ROUTER_SEED_RECORD_TYPE) throw error
+        await this.deferDecryption({
+          error, record: pendingRecord
+        })
+        continue
+      }
       if (!recovered) continue
       await this.enqueueRumor(recovered.type, channelPubkey, {
         event: recovered.event,
@@ -2847,6 +2880,8 @@ export class PrivateMessenger {
       onNymEvent: (event, outer, meta) => this.receive(channelPubkey, () => this.enqueueRumor('nym', channelPubkey, { event, outer, meta, payload: parseEventContent(event) }), { event, outer }),
       onSeedEvent: seed => this.receive(channelPubkey, () => this.enqueueSeed(channelPubkey, seed), seed),
       onContentKeyUsage: usage => this.handleContentKeyUsage(channelPubkey, usage),
+      onDecryptionFailure: failure => this.deferDecryption(failure),
+      isDecryptionPending: record => this.pendingDecryption.has(record),
       onError: err => { throw err }
     }
   }
@@ -3085,9 +3120,11 @@ export class PrivateMessenger {
     this.dropRecoveryState(pubkey)
     return this.runQueueOperation(async () => {
       this.channels.delete(pubkey)
+      this.decryptionReadyChannels?.delete(pubkey)
       this.removeChannelState(pubkey)
       await this.flushStateWrites()
       await this.queue.removeBy('byChannel', pubkey)
+      await this.pendingDecryption?.clearChannel(pubkey)
       await (this.seedStorage ? this.seedStorage.removeLocal({ channelPubkey: pubkey }) : this.seedQueue.removeBy('byChannel', pubkey))
       await this.touchStorageActivity({ force: true })
       this.ensureRelayListWatcher()
@@ -3098,6 +3135,74 @@ export class PrivateMessenger {
     return this.runQueueOperation(() => this.queue.clear())
   }
 
+  async deferDecryption (failure) {
+    this.assertOpen()
+    const pubkey = failure.record.channelPubkey
+    if (this.pauseReasons.size || !this.channels.has(pubkey)) throw this.pausedError()
+    try { return await this.pendingDecryption.save(failure) } catch (error) {
+      this.recordInterruption([pubkey], failure.record.outer?.created_at)
+      if (isRecoverableStorageFailure(error)) this.pauseInternally('storage').catch(err => this.onError?.(err))
+      throw error
+    }
+  }
+
+  retryPendingDecryptions ({ force = false } = {}) {
+    if (typeof force !== 'boolean') throw new ValidationError('INVALID_FORCE')
+    if (this.closePromise || this.pauseReasons.size || !this.initialized) return Promise.resolve()
+    if (this.decryptionRetry && !this.decryptionRetry.controller.signal.aborted) {
+      this.decryptionRetry.again = true
+      this.decryptionRetry.force ||= force
+      return this.decryptionRetry.work
+    }
+    const job = { controller: new AbortController(), again: false, force, work: null }
+    const previous = this.decryptionRetry?.work
+    this.decryptionRetry = job
+    const { signal } = job.controller
+    const run = async () => {
+      do {
+        signal.throwIfAborted()
+        job.again = false
+        const force = job.force
+        job.force = false
+        const groups = new Map()
+        for (const record of await this.pendingDecryption.records({ force })) {
+          if (!groups.has(record.channelPubkey)) groups.set(record.channelPubkey, [])
+          groups.get(record.channelPubkey).push(record)
+        }
+        for (const [channelPubkey, records] of groups) {
+          signal.throwIfAborted()
+          const channel = this.channels.get(channelPubkey)
+          if (!channel || !this.desiredChannels.has(channelPubkey)) continue
+          const current = () => { signal.throwIfAborted(); if (this.channels.get(channelPubkey) !== channel) throw new Error('PRIVATE_CHANNEL_CHANGED') }
+          await this._privateChannel.replayDeferredDecryptions({
+            records, signal, receiverSigner: this.userSigner, receiverPubkey: this.userPubkey,
+            privateChannelSigner: channel.signer, privateChannelReaderSigner: channel.readerSigner,
+            privateChannelReaderPubkey: channel.readerPubkey, mode: channel.mode,
+            receivedChunkScope: `${this.prefix}:pending`, receivedChunkIndexedDB: this._indexedDB,
+            onEvent: (event, outer, meta) => {
+              current()
+              return this.receive(channelPubkey, () => this.enqueueRumor(eventType(event), channelPubkey, { event, outer, meta, payload: parseEventContent(event) }), { event, outer })
+            },
+            onNymEvent: (event, outer, meta) => {
+              current()
+              return this.receive(channelPubkey, () => this.enqueueRumor('nym', channelPubkey, { event, outer, meta, payload: parseEventContent(event) }), { event, outer })
+            },
+            onSeedEvent: seed => { current(); return this.receive(channelPubkey, () => this.enqueueSeed(channelPubkey, seed), seed) },
+            onRecovered: record => { current(); return this.pendingDecryption.recovered(record) },
+            onFailure: (record, error) => { current(); return this.pendingDecryption.failed(record, error) }
+          })
+        }
+      } while (job.again)
+    }
+    job.work = Promise.resolve(previous).catch(() => {}).then(() => {
+      const locks = globalThis.navigator?.locks
+      return locks?.request ? locks.request(`${this.prefix}:pending-decryption:retry`, { signal }, run) : run()
+    }).catch(error => { if (!signal.aborted) throw error }).finally(() => {
+      if (this.decryptionRetry === job) this.decryptionRetry = null
+    })
+    return job.work
+  }
+
   async cleanupStaleChannels ({ storageSnapshot } = {}) {
     if (!this.prefix) return
     storageSnapshot ||= await this.readStoragePolicySnapshot()
@@ -3150,6 +3255,7 @@ export class PrivateMessenger {
   }
 
   close () {
+    this.decryptionRetry?.controller.abort(new Error('PRIVATE_MESSENGER_CLOSED'))
     if (this.closePromise) return this.closePromise
     const initSettledPromise = this.initSettledPromise
     this.sendRoutingLifetime.abort()
@@ -3189,11 +3295,13 @@ export class PrivateMessenger {
       await Promise.allSettled([...this.deliveryReads])
       await Promise.all([...this.deliveries].map(delivery => delivery.nack()))
       await this.stateWriteTail
+      await Promise.allSettled([this.decryptionRetry?.work])
       try { await this.storageTouchPromise } catch {}
       await this.storageMaintenancePromise
       try { await this.storagePolicyRefreshTail } catch {}
       await Promise.all([
         this.queue?.close?.(),
+        this.pendingDecryption?.close(),
         this.seedQueue?.close?.(),
         this.stateStore?.close?.()
       ])

package/private-messenger/services/migrate-message-queue.js

@@ -0,0 +1,95 @@
+import { deliveryInfo } from '../../private-channel/helpers/rumor.js'
+
+const encoder = new TextEncoder()
+const indexName = 'byChannelTypeEventId'
+const oldPath = ['item.channelPubkey', 'item.type', 'item.event.id']
+const newPath = [...oldPath, 'item.provenance']
+const equal = (a, b) => JSON.stringify(a) === JSON.stringify(b)
+
+function open (indexedDB, name, version, upgrade) {
+  return new Promise((resolve, reject) => {
+    const request = indexedDB.open(name, version)
+    let rejected = false
+    request.onblocked = () => { rejected = true; reject(new Error('IDB_DATABASE_BLOCKED')) }
+    request.onerror = () => reject(request.error)
+    request.onupgradeneeded = () => {
+      if (rejected) { request.transaction.abort(); return }
+      try { upgrade?.(request.result, request.transaction) } catch (error) {
+        request.transaction.abort(); reject(error)
+      }
+    }
+    request.onsuccess = () => {
+      const db = request.result
+      db.onversionchange = () => db.close()
+      if (rejected) db.close()
+      else resolve(db)
+    }
+  })
+}
+
+function matchesLegacy (items) {
+  if (!items.indexNames.contains(indexName)) return false
+  const index = items.index(indexName)
+  return equal(items.keyPath, 'position') && equal(index.keyPath, oldPath) && index.unique && !index.multiEntry &&
+    items.indexNames.contains('byChannel') && items.index('byChannel').keyPath === 'item.channelPubkey' &&
+    !items.index('byChannel').unique && !items.index('byChannel').multiEntry
+}
+
+// The generic queue remains strict. Only this historical messenger schema is
+// migrated, atomically with its records and capacity accounting.
+export async function migrateMessageQueue (indexedDB, prefix) {
+  if (!indexedDB?.open) throw new Error('IDB_UNAVAILABLE')
+  const name = `${prefix}:idb-queue`
+  for (let attempt = 0; attempt < 3; attempt++) {
+    const db = await open(indexedDB, name)
+    if (!db.objectStoreNames.contains('items') || !db.objectStoreNames.contains('state')) { db.close(); return }
+    const tx = db.transaction(['items', 'state'], 'readonly')
+    const legacy = matchesLegacy(tx.objectStore('items')) && tx.objectStore('state').keyPath === 'key'
+    const version = db.version + 1
+    await new Promise((resolve, reject) => { tx.oncomplete = resolve; tx.onabort = () => reject(tx.error) })
+    db.close()
+    if (!legacy) return
+    try {
+      const upgraded = await open(indexedDB, name, version, (_db, tx) => {
+        const items = tx.objectStore('items')
+        if (!matchesLegacy(items)) return
+        items.deleteIndex(indexName)
+        items.createIndex(indexName, newPath, { unique: true })
+        let delta = 0
+        const cursor = items.openCursor()
+        cursor.onsuccess = () => {
+          try {
+            const row = cursor.result
+            if (!row) {
+              const state = tx.objectStore('state')
+              const request = state.get('queue')
+              request.onsuccess = () => {
+                if (request.result) state.put({ ...request.result, usedBytes: request.result.usedBytes + delta })
+              }
+              return
+            }
+            const record = row.value
+            const item = { ...record.item }
+            const sender = item.senderPubkey || item.meta?.senderPubkey || item.meta?.router?.tags?.find(tag => tag[0] === 'f')?.[1] || ''
+            const info = deliveryInfo(item.event, sender)
+            item.senderPubkey = sender
+            item.provenance = item.provenance === 'hearsay' ? 'hearsay' : info.provenance
+            let byteSize = 0
+            while (true) {
+              const size = encoder.encode(JSON.stringify({ byteSize, item })).length
+              if (size === byteSize) break
+              byteSize = size
+            }
+            delta += byteSize - record.byteSize
+            row.update({ ...record, item, byteSize })
+            row.continue()
+          } catch { tx.abort() }
+        }
+      })
+      upgraded.close()
+      return
+    } catch (error) {
+      if (error?.name !== 'VersionError' || attempt === 2) throw error
+    }
+  }
+}

package/private-messenger/services/pending-decryption.js

@@ -0,0 +1,120 @@
+import { sha256 } from '@noble/hashes/sha2.js'
+import { bytesToHex } from '../../base16/index.js'
+import { base64ToBytes } from '../../base64/index.js'
+import { createQueue } from '../../idb-queue/index.js'
+import { isDeferredDecryptionError } from '../../private-channel/helpers/decryption-error.js'
+
+const encoder = new TextEncoder()
+const decoder = new TextDecoder()
+export const PENDING_DECRYPTION_MAX_BYTES = 16 * 1024 * 1024
+
+function storedBytes (item) {
+  let byteSize = 0
+  while (true) {
+    const size = encoder.encode(JSON.stringify({ byteSize, item })).length
+    if (size === byteSize) return size
+    byteSize = size
+  }
+}
+
+function packetSnapshot (packet) {
+  const event = (value, router = false) => {
+    const copy = Object.fromEntries(['id', 'kind', 'pubkey', 'created_at', 'content', 'sig'].filter(key => value?.[key] !== undefined).map(key => [key, value[key]]))
+    if (value?.tags) {
+      copy.tags = router
+        ? value.tags.filter(tag => ['f', 'p', 'imkc', 'c', 'i'].includes(tag[0])).map(tag => tag.slice(0, tag[0] === 'imkc' || tag[0] === 'c' ? 3 : 2))
+        : structuredClone(value.tags)
+    }
+    return copy
+  }
+  return { format: packet.format, channelPubkey: packet.channelPubkey, outer: event(packet.outer), ...(packet.router ? { router: event(packet.router, true) } : {}) }
+}
+
+// Stable across live, history and synthesized recovery wrappers.
+function packetId (record) {
+  const rows = record.format === 'routerRow_v1'
+    ? decoder.decode(base64ToBytes(record.router.content)).split('\n').filter(Boolean).map(line => {
+      const row = JSON.parse(line)
+      return row.length === 1 ? row : [row[0], row[1], row[2] || '', row[3] || '']
+    })
+    : null
+  const value = rows ? [record.channelPubkey, record.router.pubkey, record.router.tags.filter(tag => tag[0] !== 'c'), rows] : [record.channelPubkey, record.outer.id]
+  return bytesToHex(sha256(encoder.encode(JSON.stringify(value))))
+}
+
+export async function createPendingDecryption ({ prefix, indexedDB, maxBytes = PENDING_DECRYPTION_MAX_BYTES, report = () => {} }) {
+  const queue = await createQueue({
+    prefix: `${prefix}:pending-decryption`, indexedDB, maxBytes, evictionPolicy: 'fifo',
+    indexes: { byId: { keyPath: 'id', unique: true }, byChannel: 'channelPubkey' }
+  })
+  let tail = Promise.resolve()
+  let timer
+  let counts = {}
+  const flush = () => {
+    clearTimeout(timer); timer = null
+    if (Object.keys(counts).length) {
+      const summary = counts; counts = {}
+      report({ action: 'summary', counts: summary })
+      console.debug('private-messenger pending decryption', JSON.stringify(summary))
+    }
+  }
+  const serial = work => { const result = tail.then(work); tail = result.catch(() => {}); return result }
+  const notify = (action, detail) => {
+    report({ action, ...detail })
+    const key = `${action}:${detail.code || detail.reason || ''}`
+    counts[key] = { count: (counts[key]?.count || 0) + 1, example: detail }
+    if (!timer) { timer = setTimeout(flush, 1000); timer.unref?.() }
+  }
+  return {
+    queue,
+    has: record => serial(() => queue.someBy('byId', packetId(packetSnapshot(record)))),
+    save ({ error, record }) {
+      record = packetSnapshot(record)
+      return serial(async () => {
+        const id = packetId(record)
+        if (await queue.someBy('byId', id)) return true
+        const item = { id, channelPubkey: record.channelPubkey, packet: structuredClone(record), code: error.code, macAttempts: error.code === 'INVALID_MAC' ? 1 : 0 }
+        const requiredBytes = storedBytes(item)
+        if (requiredBytes > maxBytes) {
+          notify('discarded', { reason: 'oversized', channelPubkey: item.channelPubkey, id }); return true
+        }
+        const before = await queue.getCapacity()
+        try { await queue.enqueue(item) } catch (error) {
+          if (error.name === 'ConstraintError') return true
+          throw error
+        }
+        const after = await queue.getCapacity()
+        const evictedBytes = Math.max(0, before.usedBytes + requiredBytes - after.usedBytes)
+        if (evictedBytes) notify('discarded', { reason: 'capacity', bytes: evictedBytes })
+        const coordinates = Object.fromEntries(['eventId', 'eventKind', 'senderPubkey', 'receiverPubkey', 'ownContentPubkey', 'peerContentPubkey', 'hasOwnContentKey']
+          .filter(key => ['string', 'boolean', 'number'].includes(typeof error[key])).map(key => [key, error[key]]))
+        notify('deferred', { ...coordinates, code: error.code, stage: error.stage, id, channelPubkey: item.channelPubkey, capacityBytesBefore: before.usedBytes, capacityBytesAfter: after.usedBytes })
+        return true
+      })
+    },
+    async records ({ force = false } = {}) {
+      await tail
+      const records = []
+      for await (const item of queue.storedItems()) if (force || item.macAttempts < 3) records.push(item)
+      return records
+    },
+    recovered (item) {
+      return serial(async () => {
+        await queue.removeBy('byId', item.id)
+        notify('recovered', { id: item.id, channelPubkey: item.channelPubkey })
+      })
+    },
+    failed (item, error) {
+      return serial(async () => {
+        if (!isDeferredDecryptionError(error)) throw error
+        const current = await queue.getBy('byId', item.id)
+        if (!current) return
+        const updated = { ...current, code: error.code, macAttempts: current.macAttempts + (error.code === 'INVALID_MAC' ? 1 : 0) }
+        await queue.putBy('byId', updated, { existingOnly: true })
+        if (updated.macAttempts >= 3 && item.macAttempts < 3) notify('suspended', { id: item.id, code: error.code, channelPubkey: item.channelPubkey })
+      })
+    },
+    clearChannel: channel => serial(() => queue.removeBy('byChannel', channel)),
+    async close () { await tail; flush(); await queue.close() }
+  }
+}

package/private-messenger/services/storage-maintenance.js

@@ -185,6 +185,7 @@ function storageDatabaseNames (userPubkey) {
   const prefix = `libp2r2p:private-messenger:${userPubkey}`
   return [
     `${prefix}:idb-queue`,
+    `${prefix}:pending-decryption:idb-queue`,
     `${prefix}:seeds:idb-queue`,
     `${prefix}:file-seeds:idb-queue`,
     `${prefix}:file-authorizations:idb-queue`,

package/private-messenger/session/README.md

@@ -4,6 +4,12 @@
 outbox and IRFS attachments without presentation state. It builds on
 PrivateMessenger and the file coordinator, not another recovery protocol.
 
+`retryPendingDecryptions({ force = false } = {})` forwards a background pass to
+the account messenger while available. Call after committed content-key changes;
+ordinary availability recovery already requests a pass. Ciphertext stays in the
+separate bounded FIFO until admitted to the normal inbox, evicted or explicitly
+cleared. This does not retry application storage/permission failures as crypto.
+
 Provide `owner`, a byte-based NIP-44 v3 `signer` with `withSharedKey`, and either
 `eventStore` or a `messageStorage.save(event, { peerPubkey, hearsay })` adapter.
 Optional `chunkStorage` and `recoveryStorage` (`seeds`, `authorizations`) select

package/private-messenger/session/index.js

@@ -558,6 +558,11 @@ export function createPrivateMessageSession ({ owner, signer, eventStore, messag
     await writeDownloadIntent(() => downloads?.remove(id))
   }
   return {
+    async retryPendingDecryptions (options) {
+      if (closed || !available) return
+      await ensureMessenger()
+      if (!closed && available) return messenger?.retryPendingDecryptions(options)
+    },
     download, cancelDownload, preparePeer,
     // Peers-based so callers can prefetch as soon as a chat route opens, before
     // the shared-key channel exists. When group chats get real membership, add