@beam-network/sdk 0.6.0 → 0.6.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -156,19 +156,19 @@ For low-level raw transfer configs, use `createRawTransfer`; lifecycle transport
156
156
 
157
157
  Provider transfers use `transfer-client-control/v7`. Every reply carries Runtime and transport epochs; prepare and every route-stream message carry a UUID route generation. S3, R2, and S3-compatible destinations retain direct multipart UploadPart/ListParts/HEAD handling. Hippius and Hugging Face destinations take a plain PUT per chunk instead, so they carry no multipart group manifest.
158
158
 
159
- The client signs up to 64 routes concurrently by default and emits 2,048-route logical batches. Encoded MessagePack requests target 4 MiB physical batches under `maxPayloadBytes` (8 MiB by default; explicit positive overrides remain supported). The splitter reserves control-envelope headroom for the live auth token and stable request identity so the final encoded request remains below the configured guard. Lifecycle control refreshes cached SDK auth before the final 30 seconds of token lifetime and retries `auth_token_expired` replies with the same request identity and payload but a fresh token, preserving Core request-conflict protection during large route streams. A single route above the 4 MiB target but within the configured guard publishes alone; a single route larger than the configured limit fails before publication with both sizes in the error. Multipart create and abort requests use a separate `multipartControlConcurrency` limit of 2 by default, with explicit positive overrides supported, and S3-compatible control requests make up to five attempts for transient provider failures. This keeps URL signing throughput independent from provider control-plane pressure.
159
+ The client signs up to 64 routes concurrently by default and emits 1,024-route logical batches. Encoded MessagePack requests target 4 MiB physical batches under `maxPayloadBytes` (8 MiB by default; explicit positive overrides remain supported). The splitter reserves control-envelope headroom for the live auth token and stable request identity so the final encoded request remains below the configured guard. Lifecycle control refreshes cached SDK auth before the final 30 seconds of token lifetime and retries `auth_token_expired` replies with the same request identity and payload but a fresh token, preserving Core request-conflict protection during large route streams. A single route above the 4 MiB target but within the configured guard publishes alone; a single route larger than the configured limit fails before publication with both sizes in the error. Multipart create and abort requests use a separate `multipartControlConcurrency` limit of 2 by default, with explicit positive overrides supported, and S3-compatible control requests make up to five attempts for transient provider failures. This keeps URL signing throughput independent from provider control-plane pressure.
160
160
 
161
161
  Callers that need durable recovery accounting can provide `onMultipartGroupReady`. The SDK invokes it after creating a multipart upload and before streaming that group’s signed routes. Its payload contains transfer, source, destination, object-key, upload, size, part-count, and expiry identities only; it never includes credentials, request headers, or signed URLs. A rejected callback aborts the newly created upload and fails preparation closed.
162
162
 
163
163
  `resumeProviderTransfer` requires the complete saved callback payloads in `multipartGroups`. It validates every transfer/source/destination/object/size/part coordinate, then uses the same route-stream implementation as creation with those upload IDs. Missing, duplicate, or changed identities fail before new provider uploads or route publication. If a process died before recording an upload identity, treat it as uncertain cleanup instead of inventing a replacement. The `onPrepared` callback also runs during resume, before routes are streamed.
164
164
 
165
- Create and resume accept an ownership `signal`. Aborting it stops initial routes, multipart creation, and recovery signing/replay; it does not cancel the transfer or abort a replacement owner's uploads. User cancellation still uses `cancelTransfer` and verified provider cleanup. `prepareProviderSource` accepts the same signal for metadata requests.
165
+ Create and resume accept an ownership `signal`. Aborting it stops initial routes, multipart creation, and recovery signing/replay; it does not cancel the transfer or abort a replacement owner's uploads. Recovery leases, signers, and background resume/replay are keyed by owner identity: a fenced-off owner never replays into, releases, or stops the signers of a replacement owner registered for the same transfer, and a replacement always gets its own recovery. User cancellation still uses `cancelTransfer` and verified provider cleanup. `prepareProviderSource` accepts the same signal for metadata requests.
166
166
 
167
167
  The stream ID derives from the transfer, selected flow, and immutable plan identity, and each ordered batch ID includes its route-coordinate checksum. Manual multi-destination attachment requires `delivery_index` on every route. Lifecycle mutations retry transient failures up to three times with the same request identity. A non-recoverable provider setup or route failure raises `BeamProviderTransferError`, whose `errors`, `transferCancelled`, and `multipartCleanupComplete` fields retain the original provider, cancellation, and cleanup outcomes. `waitForTransfer` subscribes to the API-key-owned, at-most-once terminal signal before its first status read and reconciles every signal through authoritative status; subscription failure degrades to the same jittered 15-to-30-second status fallback. Call `close()` when the client is no longer needed; it also closes outstanding terminal waiters.
168
168
 
169
169
  During the pre-completion `integrity_check` phase, Runtime may include an `integrity_audit_challenge` in status. The TypeScript client signs exact read-only source and final-destination GET ranges and submits `transfer.integrity_audit_grants`. S3-compatible source and destination grants use the prepared source ETag and Runtime-verified final-object ETag as `If-Match` conditions. Submission failures retry on the next status poll and appear as a URL-safe `integrity_audit_submission_error` on that status response; the transfer completes with a warning if the check cannot finish by its deadline.
170
170
 
171
- Hippius uses canonical non-multipart `signed_url` routes, so its manifest is empty and group-level final HEAD verification is not available from that provider flow.
171
+ Hippius uses canonical non-multipart `signed_url` routes (initial and recovered routes both PUT to the planned `<final>/<plan_nonce>/chunk-NNNNNN` key), so its manifest is empty and group-level final HEAD verification is not available from that provider flow.
172
172
 
173
173
  NATS requires this guard because the broker rejects messages above its configured `max_payload`. This limit applies only to lifecycle/control metadata; transfer file bytes do not flow through NATS.
174
174
 
@@ -189,3 +189,9 @@ Only short-lived range/upload routes go to workers. Multipart creation, verifica
189
189
  Manual releases from `dev` require a committed `-dev.N` version and publish only to npm tag `dev`; the stable tag is unchanged.
190
190
 
191
191
  Multipart control helpers (`createMultipartUpload`, `listMultipartParts`, `completeMultipartUpload`, `inspectDestinationObject`, and `abortMultipartUpload`) accept an optional `AbortSignal` and cancel through the existing provider HTTP transport. Pass it as `signal` in object arguments or as the last positional argument. Cancellation stops waiting and network activity; it cannot undo a provider operation already accepted. After an interrupted completion, inspect the durable upload/object identity before retrying or reporting cleanup complete.
192
+
193
+ ### Preparation diagnostics
194
+
195
+ Route streaming flushes at 1,024 routes, with no timer, and sends the final partial batch at stream completion. Signing overlaps acknowledged publication through bounded buffering; existing encoded-payload limits still apply. A source grant is reused across destinations only within the same signing generation and actual expiry.
196
+
197
+ The optional `onDiagnostics` callback receives bounded preparation measurements and source-grant reuse counts. Callbacks are best effort, may be dropped under load, and do not affect transfer outcomes. Durations may overlap; do not sum them into elapsed transfer time. Transfer status also exposes a typed `performance` summary when supported by Core.
package/dist/client.d.ts CHANGED
@@ -44,8 +44,11 @@ export declare class BeamClient {
44
44
  private readonly routeSigningConcurrency;
45
45
  private readonly routeSigningConcurrencyOverridden;
46
46
  private readonly multipartControlConcurrency;
47
+ private readonly onDiagnostics;
48
+ private diagnosticPending;
47
49
  private readonly recoverySigners;
48
50
  private readonly integrityAuditSigners;
51
+ private readonly integrityGrantCache;
49
52
  private readonly integrityAuditSubmissions;
50
53
  private readonly huggingFaceUploads;
51
54
  constructor(options?: BeamClientOptions);
@@ -105,7 +108,9 @@ export declare class BeamClient {
105
108
  private startProviderRouteRecoverySigner;
106
109
  private recoveryMultipartUploadState;
107
110
  private submitIntegrityAuditGrantsIfPresent;
108
- private submitProviderIntegrityAuditGrants;
111
+ private signedIntegrityGrants;
112
+ private buildProviderIntegrityAuditGrants;
113
+ /** Stop a transfer's recovery and integrity signers; with `owner`, only if that owner installed them. */
109
114
  private stopRecoverySigner;
110
115
  private stopAllRecoverySigners;
111
116
  createAndDistribute(input: RawTransferCreateInput): Promise<TransferCreateResponse>;
package/dist/client.js CHANGED
@@ -1,4 +1,5 @@
1
- import { signMultipartRecovery, abortMultipartUpload, createMultipartUpload, expiresAtIso, prepareProviderDestination, prepareProviderSource, signAbortMultipartUpload, signCompleteMultipartUpload, signDestinationReadRange, signDestinationRoute, signFinalObjectHead, signListMultipartUpload, signSourceReadRange, isHuggingFaceProvider } from "./provider-signing.js";
1
+ import { SdkPerformanceCollector } from "./performance.js";
2
+ import { signMultipartRecovery, abortMultipartUpload, createMultipartUpload, expiresAtIso, prepareProviderDestination, prepareProviderSource, releaseProviderClients, signAbortMultipartUpload, signCompleteMultipartUpload, signDestinationReadRange, signDestinationRoute, signFinalObjectHead, signListMultipartUpload, signSourceReadRange, signSourceChunk, boundedGrantExpiry, isHuggingFaceProvider } from "./provider-signing.js";
2
3
  import { describe as describeHuggingFace, hashSourceStream, huggingFaceCommit, huggingFaceCompleteLfsUpload, huggingFaceLfsBatch, huggingFacePreupload, huggingFaceVerifyLfsUpload, readSourceSample } from "./huggingface.js";
3
4
  import { BeamTransferControl, BEAM_DEFAULT_NATS_URL, compactSignedRoutes, isRecoverableRouteStreamError } from "./nats-control.js";
4
5
  export class BeamRouteRecoveryPendingError extends Error {
@@ -53,11 +54,15 @@ export class BeamClient {
53
54
  routeSigningConcurrency;
54
55
  routeSigningConcurrencyOverridden;
55
56
  multipartControlConcurrency;
57
+ onDiagnostics;
58
+ diagnosticPending = false;
56
59
  recoverySigners = new Map();
57
60
  integrityAuditSigners = new Map();
61
+ integrityGrantCache = new Map();
58
62
  integrityAuditSubmissions = new Map();
59
63
  huggingFaceUploads = new Map();
60
64
  constructor(options = {}) {
65
+ this.onDiagnostics = options.onDiagnostics;
61
66
  if (!options.apiKey?.trim()) {
62
67
  throw new Error("apiKey is required.");
63
68
  }
@@ -237,7 +242,7 @@ export class BeamClient {
237
242
  };
238
243
  const routeStreamLock = new AsyncMutex();
239
244
  const releaseInitialStream = await routeStreamLock.acquire();
240
- this.control.registerRecoveryLease({
245
+ const recoveryLease = {
241
246
  transferId,
242
247
  planFingerprint: input.planFingerprint,
243
248
  coordinateChecksum: input.coordinateChecksum,
@@ -249,19 +254,20 @@ export class BeamClient {
249
254
  throw new Error(attached.error ?? attached.message ?? "route replay failed");
250
255
  });
251
256
  }
252
- });
257
+ };
258
+ this.control.registerRecoveryLease(recoveryLease);
253
259
  try {
254
260
  const attached = await streamRoutes(input.chunkRoutes, input.multipartGroupManifest, input.routeGenerationId, input.urlsExpiresAt);
255
261
  if (!attached.success)
256
- this.control.releaseRecoveryLease(transferId);
262
+ this.control.releaseRecoveryLease(transferId, recoveryLease);
257
263
  return attached;
258
264
  }
259
265
  catch (error) {
260
266
  if (isRecoverableRouteStreamError(error)) {
261
- this.control.continueRecoveryLease(transferId);
267
+ this.control.continueRecoveryLease(transferId, recoveryLease);
262
268
  throw new BeamRouteRecoveryPendingError(transferId, error);
263
269
  }
264
- this.control.releaseRecoveryLease(transferId);
270
+ this.control.releaseRecoveryLease(transferId, recoveryLease);
265
271
  throw error;
266
272
  }
267
273
  finally {
@@ -294,8 +300,10 @@ export class BeamClient {
294
300
  destinationsById.set(preparedDestination.destination_id, destination);
295
301
  });
296
302
  await input.throwIfCancelled?.();
303
+ const discoveryStarted = performance.now();
297
304
  const preparedSources = await Promise.all(retainedRecoveryInput.sources.map((source, index) => prepareProviderSource(source, { index, expiresIn, fetchImpl: this.fetchImpl, signal: input.signal })));
298
305
  await input.throwIfCancelled?.();
306
+ const discoveryMs = performance.now() - discoveryStarted;
299
307
  const huggingFace = await this.planHuggingFaceUploads({
300
308
  sources: retainedRecoveryInput.sources,
301
309
  preparedSources,
@@ -346,6 +354,9 @@ export class BeamClient {
346
354
  const routeStreamLock = new AsyncMutex();
347
355
  const releaseInitialStream = await routeStreamLock.acquire();
348
356
  const streamPreparedRoutes = async (routeGenerationId, recoveryReplay) => {
357
+ const telemetry = new SdkPerformanceCollector();
358
+ if (!recoveryReplay)
359
+ telemetry.observe("sdk.discovery", discoveryMs);
349
360
  const pendingRoutes = new Set();
350
361
  let signingConcurrency = this.routeSigningConcurrency;
351
362
  let signedInWindow = 0;
@@ -376,6 +387,15 @@ export class BeamClient {
376
387
  signedUrlFlow: requestedSignedUrlFlow
377
388
  });
378
389
  routeStream = new RouteStreamSender(this.control, {
390
+ telemetry,
391
+ onDiagnostics: (summary) => {
392
+ if (!this.onDiagnostics || this.diagnosticPending)
393
+ return;
394
+ this.diagnosticPending = true;
395
+ setTimeout(() => {
396
+ void Promise.resolve().then(() => this.onDiagnostics?.(summary)).catch(() => { }).finally(() => { this.diagnosticPending = false; });
397
+ }, 0);
398
+ },
379
399
  streamId,
380
400
  transferId: prepared.transfer_id,
381
401
  routeGenerationId,
@@ -388,7 +408,7 @@ export class BeamClient {
388
408
  routeStreamBeginAttempted = true;
389
409
  await routeStream.begin();
390
410
  multipartGroupWaiters = createMultipartGroupWaiters(prepared, destinationsById);
391
- multipartManifestTask = createMultipartGroupManifest({
411
+ multipartManifestTask = telemetry.measure("sdk.multipart_create", () => createMultipartGroupManifest({
392
412
  prepared,
393
413
  destinationsById,
394
414
  multipartUploads,
@@ -405,7 +425,7 @@ export class BeamClient {
405
425
  multipartGroupWaiters.get(state.manifest.multipart_group_id)?.resolve(state);
406
426
  },
407
427
  onGroupFailed: (groupId, error) => multipartGroupWaiters.get(groupId)?.reject(error)
408
- });
428
+ }));
409
429
  const streamNextCompletedRoute = async () => {
410
430
  const settled = await Promise.race([...pendingRoutes].map((pending) => pending.then((route) => ({ pending, route }))));
411
431
  pendingRoutes.delete(settled.pending);
@@ -423,6 +443,12 @@ export class BeamClient {
423
443
  };
424
444
  for (const chunk of materializePlanChunks(prepared.plan_descriptor, prepared.transfer_id)) {
425
445
  await throwIfCancelled?.(prepared.transfer_id);
446
+ // This promise belongs to this chunk and signing generation only. Its
447
+ // immutable credential/range identity and actual expiry cannot be renewed.
448
+ const sourceGrant = signSourceChunk({ source: sourcesById.get(chunk.source_id), fallbackUrl: chunk.source_url, chunk, expiresIn, fetchImpl: this.fetchImpl });
449
+ void sourceGrant.catch(() => { });
450
+ telemetry.counters.source_signatures++;
451
+ telemetry.counters.source_reuses += Math.max(0, chunk.destinations.length - 1);
426
452
  for (const target of chunk.destinations) {
427
453
  const pendingRoute = (async () => {
428
454
  await throwIfCancelled?.(prepared.transfer_id);
@@ -442,10 +468,11 @@ export class BeamClient {
442
468
  : await multipartGroupWaiters.get(multipartGroupId)?.promise;
443
469
  if (!isDirectPutDestination(destination) && !upload)
444
470
  throw new Error(`multipart group manifest is missing for ${chunk.source_id}:${target.destination_id}`);
445
- return this.signProviderRoute({
471
+ return telemetry.measure("sdk.signing", () => this.signProviderRoute({
446
472
  chunk,
447
473
  target,
448
474
  source,
475
+ sourceGrant,
449
476
  destination,
450
477
  expiresIn,
451
478
  ...(upload ? { upload } : {}),
@@ -456,7 +483,7 @@ export class BeamClient {
456
483
  partNumber: typeof metadata.part_number === "number"
457
484
  ? metadata.part_number
458
485
  : multipartPartNumber(chunk.source_chunk_index),
459
- });
486
+ }));
460
487
  })();
461
488
  pendingRoutes.add(pendingRoute);
462
489
  if (pendingRoutes.size >= signingConcurrency)
@@ -475,8 +502,8 @@ export class BeamClient {
475
502
  }
476
503
  catch (error) {
477
504
  if (input.signal?.aborted) {
478
- this.stopRecoverySigner(prepared.transfer_id);
479
- this.control.releaseRecoveryLease(prepared.transfer_id);
505
+ this.stopRecoverySigner(prepared.transfer_id, recoveryLease);
506
+ this.control.releaseRecoveryLease(prepared.transfer_id, recoveryLease);
480
507
  await Promise.allSettled(pendingRoutes);
481
508
  if (multipartManifestTask)
482
509
  await Promise.allSettled([multipartManifestTask]);
@@ -489,10 +516,10 @@ export class BeamClient {
489
516
  await Promise.allSettled([multipartManifestTask]);
490
517
  if (!recoveryReplay) {
491
518
  if (foregroundCancelled) {
492
- this.control.continueRecoveryLease(prepared.transfer_id);
519
+ this.control.continueRecoveryLease(prepared.transfer_id, recoveryLease);
493
520
  }
494
521
  else if (isRecoverableRouteStreamError(error)) {
495
- this.control.continueRecoveryLease(prepared.transfer_id);
522
+ this.control.continueRecoveryLease(prepared.transfer_id, recoveryLease);
496
523
  throw new BeamRouteRecoveryPendingError(prepared.transfer_id, error);
497
524
  }
498
525
  else {
@@ -510,27 +537,16 @@ export class BeamClient {
510
537
  // The retained recovery input owns the same destination objects used by
511
538
  // multipart cleanup. Releasing it earlier scrubs their scoped credentials
512
539
  // before abortMultipartUpload can clean up already-created uploads.
513
- this.control.releaseRecoveryLease(prepared.transfer_id);
540
+ this.control.releaseRecoveryLease(prepared.transfer_id, recoveryLease);
514
541
  }
515
542
  }
516
543
  }
517
544
  throw error;
518
545
  }
519
546
  };
520
- await this.startProviderRouteRecoverySigner({
521
- prepared,
522
- sourcesById,
523
- destinationsById,
524
- multipartUploads: recoveryMultipartUploads,
525
- signal: input.signal,
526
- expiresIn
527
- });
528
- const stopOwnedRecovery = () => {
529
- this.stopRecoverySigner(prepared.transfer_id);
530
- this.control.releaseRecoveryLease(prepared.transfer_id);
531
- };
532
- input.signal?.addEventListener("abort", stopOwnedRecovery, { once: true });
533
- this.control.registerRecoveryLease({
547
+ // The lease object is this owner's identity: releases, recovery requests, and
548
+ // signer shutdowns below only act while it is still the registered lease.
549
+ const recoveryLease = {
534
550
  transferId: prepared.transfer_id,
535
551
  planFingerprint: prepared.plan_fingerprint,
536
552
  coordinateChecksum: prepared.coordinate_checksum,
@@ -539,23 +555,39 @@ export class BeamClient {
539
555
  },
540
556
  disposeSecrets: () => {
541
557
  input.signal?.removeEventListener("abort", stopOwnedRecovery);
558
+ releaseProviderClients([...retainedRecoveryInput.sources, ...retainedRecoveryInput.destinations]);
542
559
  clearRecoverySecrets(retainedRecoveryInput);
543
560
  }
561
+ };
562
+ await this.startProviderRouteRecoverySigner({
563
+ owner: recoveryLease,
564
+ prepared,
565
+ sourcesById,
566
+ destinationsById,
567
+ multipartUploads: recoveryMultipartUploads,
568
+ signal: input.signal,
569
+ expiresIn
544
570
  });
571
+ const stopOwnedRecovery = () => {
572
+ this.stopRecoverySigner(prepared.transfer_id, recoveryLease);
573
+ this.control.releaseRecoveryLease(prepared.transfer_id, recoveryLease);
574
+ };
575
+ input.signal?.addEventListener("abort", stopOwnedRecovery, { once: true });
576
+ this.control.registerRecoveryLease(recoveryLease);
545
577
  try {
546
578
  try {
547
579
  assertOwnership();
548
580
  await input.onPrepared?.(prepared);
549
581
  }
550
582
  catch (error) {
551
- this.control.continueRecoveryLease(prepared.transfer_id);
583
+ this.control.continueRecoveryLease(prepared.transfer_id, recoveryLease);
552
584
  throw error;
553
585
  }
554
586
  try {
555
587
  await input.throwIfCancelled?.(prepared.transfer_id);
556
588
  }
557
589
  catch (error) {
558
- this.control.continueRecoveryLease(prepared.transfer_id);
590
+ this.control.continueRecoveryLease(prepared.transfer_id, recoveryLease);
559
591
  throw error;
560
592
  }
561
593
  await streamPreparedRoutes(prepared.route_generation_id, false);
@@ -740,6 +772,7 @@ export class BeamClient {
740
772
  metadata: directPutRouteMetadata(input.target.metadata)
741
773
  },
742
774
  source: input.source,
775
+ sourceGrant: input.sourceGrant,
743
776
  destination: input.destination,
744
777
  expiresIn: input.expiresIn,
745
778
  destUrl,
@@ -751,6 +784,7 @@ export class BeamClient {
751
784
  chunk: input.chunk,
752
785
  target: { ...input.target, metadata: partRouteMetadata(input.target.metadata) },
753
786
  source: input.source,
787
+ sourceGrant: input.sourceGrant,
754
788
  destination: input.destination,
755
789
  expiresIn: input.expiresIn,
756
790
  fetchImpl: this.fetchImpl
@@ -768,6 +802,7 @@ export class BeamClient {
768
802
  metadata: partRouteMetadata(input.target.metadata)
769
803
  },
770
804
  source: input.source,
805
+ sourceGrant: input.sourceGrant,
771
806
  destination: input.destination,
772
807
  expiresIn: input.expiresIn,
773
808
  uploadId: upload.uploadId,
@@ -791,6 +826,7 @@ export class BeamClient {
791
826
  input.signal?.throwIfAborted();
792
827
  if (payload.route_generation_id.length === 0)
793
828
  throw new Error("route recovery generation is required");
829
+ const sourceGrants = new Map();
794
830
  const routes = await mapOrderedWithConcurrency(payload.chunks, this.routeSigningConcurrency, async (requested) => {
795
831
  input.signal?.throwIfAborted();
796
832
  if (requested.route_generation_id !== payload.route_generation_id) {
@@ -821,11 +857,22 @@ export class BeamClient {
821
857
  multipartUploads: input.multipartUploads,
822
858
  expiresIn: input.expiresIn
823
859
  });
860
+ const sourceKey = JSON.stringify([chunk.source_id, chunk.chunk_index]);
861
+ let sourceGrant = sourceGrants.get(sourceKey);
862
+ if (!sourceGrant) {
863
+ sourceGrant = signSourceChunk({ source, chunk, fallbackUrl: chunk.source_url, expiresIn: input.expiresIn, fetchImpl: this.fetchImpl });
864
+ sourceGrants.set(sourceKey, sourceGrant);
865
+ void sourceGrant.catch(() => { });
866
+ }
824
867
  const route = await this.signProviderRoute({
868
+ sourceGrant,
825
869
  chunk,
826
870
  target: {
827
871
  ...target,
828
- object_key: requested.final_object_key,
872
+ // Direct-PUT destinations (Hippius, Hugging Face) keep the planned
873
+ // per-chunk key exactly as initial materialization does; only
874
+ // multipart destinations sign parts against the final object key.
875
+ object_key: isDirectPutDestination(destination) ? target.object_key : requested.final_object_key,
829
876
  metadata: {
830
877
  ...(target.metadata ?? {}),
831
878
  ...(requested.destination_metadata ?? {}),
@@ -861,16 +908,33 @@ export class BeamClient {
861
908
  chunk_routes: routes
862
909
  };
863
910
  });
864
- this.recoverySigners.set(input.prepared.transfer_id, stop);
865
- this.integrityAuditSigners.set(input.prepared.transfer_id, async (challenge) => {
866
- await this.submitProviderIntegrityAuditGrants({
867
- challenge,
868
- prepared: input.prepared,
869
- sourcesById: input.sourcesById,
870
- destinationsById: input.destinationsById,
871
- expiresIn: input.expiresIn
911
+ if (input.signal?.aborted) {
912
+ stop();
913
+ return;
914
+ }
915
+ this.stopRecoverySigner(input.prepared.transfer_id);
916
+ this.integrityAuditSigners.set(input.prepared.transfer_id, (challenge) => this.buildProviderIntegrityAuditGrants({
917
+ challenge, prepared: input.prepared, sourcesById: input.sourcesById, destinationsById: input.destinationsById, expiresIn: input.expiresIn
918
+ }));
919
+ const registration = { owner: input.owner, stop };
920
+ this.recoverySigners.set(input.prepared.transfer_id, registration);
921
+ try {
922
+ const stopIntegrity = await this.control.serveIntegritySigner(input.prepared.transfer_id, challenge => {
923
+ input.signal?.throwIfAborted();
924
+ return this.signedIntegrityGrants(challenge);
872
925
  });
873
- });
926
+ if (input.signal?.aborted || this.recoverySigners.get(input.prepared.transfer_id) !== registration) {
927
+ stop();
928
+ stopIntegrity();
929
+ return;
930
+ }
931
+ registration.stop = () => { stop(); stopIntegrity(); };
932
+ }
933
+ catch {
934
+ // Status-driven signing remains available if the optional subscription fails.
935
+ if (input.signal?.aborted)
936
+ this.stopRecoverySigner(input.prepared.transfer_id, input.owner);
937
+ }
874
938
  }
875
939
  async recoveryMultipartUploadState(input) {
876
940
  const existing = input.multipartUploads.get(input.requested.multipart_group_id);
@@ -937,21 +1001,56 @@ export class BeamClient {
937
1001
  if (!signer)
938
1002
  throw new Error("integrity audit signer unavailable");
939
1003
  const existing = this.integrityAuditSubmissions.get(challenge.audit_id);
940
- if (existing) {
1004
+ const cached = this.integrityGrantCache.get(challenge.audit_id);
1005
+ if (cached && cached.fingerprint !== JSON.stringify(challenge))
1006
+ throw new Error("integrity audit identity changed");
1007
+ if (existing && cached && cached.expiresAt > Date.now() + 30_000) {
941
1008
  await existing;
942
1009
  return;
943
1010
  }
944
- const submission = signer(challenge);
1011
+ const submission = this.signedIntegrityGrants(challenge).then(async (payload) => {
1012
+ const result = await this.control.request("transfer.integrity_audit_grants", payload, {
1013
+ transferId: challenge.transfer_id, idempotencyKey: `transfer:${challenge.transfer_id}:integrity-audit:${challenge.audit_id}:${payload.submitted_at}`
1014
+ });
1015
+ if (result.published !== true)
1016
+ throw new Error("integrity audit delivery unavailable");
1017
+ });
945
1018
  this.integrityAuditSubmissions.set(challenge.audit_id, submission);
946
1019
  try {
947
1020
  await submission;
948
1021
  }
949
1022
  catch (error) {
950
- this.integrityAuditSubmissions.delete(challenge.audit_id);
1023
+ if (this.integrityAuditSubmissions.get(challenge.audit_id) === submission)
1024
+ this.integrityAuditSubmissions.delete(challenge.audit_id);
951
1025
  throw error;
952
1026
  }
953
1027
  }
954
- async submitProviderIntegrityAuditGrants(input) {
1028
+ signedIntegrityGrants(challenge) {
1029
+ const fingerprint = JSON.stringify(challenge);
1030
+ const cached = this.integrityGrantCache.get(challenge.audit_id);
1031
+ if (cached && cached.fingerprint !== fingerprint)
1032
+ return Promise.reject(new Error("integrity audit identity changed"));
1033
+ if (cached && cached.expiresAt > Date.now() + 30_000)
1034
+ return cached.payload;
1035
+ if (!cached && this.integrityGrantCache.size >= 1024)
1036
+ return Promise.reject(new Error("integrity signer capacity unavailable"));
1037
+ const signer = this.integrityAuditSigners.get(challenge.transfer_id);
1038
+ if (!signer)
1039
+ return Promise.reject(new Error("integrity audit signer unavailable"));
1040
+ this.integrityAuditSubmissions.delete(challenge.audit_id);
1041
+ const entry = { transferId: challenge.transfer_id, expiresAt: 0, fingerprint, payload: Promise.resolve({}) };
1042
+ entry.payload = signer(challenge).then(payload => {
1043
+ const chunks = payload.chunks;
1044
+ entry.expiresAt = Math.min(...chunks.flatMap(chunk => [Date.parse(chunk.source.expires_at), Date.parse(chunk.destination.expires_at)]));
1045
+ return payload;
1046
+ }).catch(error => { if (this.integrityGrantCache.get(challenge.audit_id) === entry)
1047
+ this.integrityGrantCache.delete(challenge.audit_id); throw error; });
1048
+ // Pending grants are shared too. Actual expiry replaces this sentinel on completion.
1049
+ entry.expiresAt = Infinity;
1050
+ this.integrityGrantCache.set(challenge.audit_id, entry);
1051
+ return entry.payload;
1052
+ }
1053
+ async buildProviderIntegrityAuditGrants(input) {
955
1054
  if (input.challenge.transfer_id !== input.prepared.transfer_id) {
956
1055
  throw new Error("integrity audit challenge transfer mismatch");
957
1056
  }
@@ -978,6 +1077,7 @@ export class BeamClient {
978
1077
  }
979
1078
  const sourceEtag = plannedSource?.metadata?.etag;
980
1079
  const sourceVersionId = plannedSource?.metadata?.version_id;
1080
+ const grantExpiresAt = new Date(Math.floor(Date.now() / 1000) * 1000 + input.expiresIn * 1000).toISOString();
981
1081
  const [sourceGrant, destinationGrant] = await Promise.all([
982
1082
  signSourceReadRange({
983
1083
  source,
@@ -998,31 +1098,30 @@ export class BeamClient {
998
1098
  fetchImpl: this.fetchImpl
999
1099
  })
1000
1100
  ]);
1001
- const grantExpiresAt = expiresAtIso(input.expiresIn);
1002
1101
  const { orchestrator_id: _orchestratorId, orchestrator_hotkey: _orchestratorHotkey, worker_id: _workerId, ...grantChunk } = chunk;
1003
1102
  return {
1004
1103
  ...grantChunk,
1005
- source: { ...sourceGrant, expires_at: grantExpiresAt },
1006
- destination: { ...destinationGrant, expires_at: grantExpiresAt }
1104
+ source: { ...sourceGrant, expires_at: boundedGrantExpiry(sourceGrant.url, grantExpiresAt) },
1105
+ destination: { ...destinationGrant, expires_at: boundedGrantExpiry(destinationGrant.url, grantExpiresAt) }
1007
1106
  };
1008
1107
  });
1009
- await this.control.request("transfer.integrity_audit_grants", {
1010
- transfer_id: input.challenge.transfer_id,
1011
- audit_id: input.challenge.audit_id,
1012
- submitted_at: submittedAt,
1013
- chunks
1014
- }, {
1015
- transferId: input.challenge.transfer_id,
1016
- idempotencyKey: `transfer:${input.challenge.transfer_id}:integrity-audit:${input.challenge.audit_id}`
1017
- });
1108
+ return { transfer_id: input.challenge.transfer_id, audit_id: input.challenge.audit_id, submitted_at: submittedAt, chunks };
1018
1109
  }
1019
- stopRecoverySigner(transferId) {
1020
- const stop = this.recoverySigners.get(transferId);
1021
- if (stop) {
1022
- stop();
1110
+ /** Stop a transfer's recovery and integrity signers; with `owner`, only if that owner installed them. */
1111
+ stopRecoverySigner(transferId, owner) {
1112
+ const signer = this.recoverySigners.get(transferId);
1113
+ if (owner && signer?.owner !== owner)
1114
+ return;
1115
+ if (signer) {
1116
+ signer.stop();
1023
1117
  this.recoverySigners.delete(transferId);
1024
1118
  }
1025
1119
  this.integrityAuditSigners.delete(transferId);
1120
+ for (const [auditId, entry] of this.integrityGrantCache)
1121
+ if (entry.transferId === transferId) {
1122
+ this.integrityGrantCache.delete(auditId);
1123
+ this.integrityAuditSubmissions.delete(auditId);
1124
+ }
1026
1125
  }
1027
1126
  stopAllRecoverySigners() {
1028
1127
  for (const transferId of [...this.recoverySigners.keys()])
@@ -1118,7 +1217,7 @@ export class BeamClient {
1118
1217
  }
1119
1218
  }
1120
1219
  }
1121
- const ROUTE_STREAM_BATCH_ROUTES = 2_048;
1220
+ const ROUTE_STREAM_BATCH_ROUTES = 1_024;
1122
1221
  class AsyncMutex {
1123
1222
  tail = Promise.resolve();
1124
1223
  async acquire() {
@@ -1143,6 +1242,8 @@ class AsyncMutex {
1143
1242
  class RouteStreamSender {
1144
1243
  control;
1145
1244
  options;
1245
+ started = performance.now();
1246
+ telemetry;
1146
1247
  streamId;
1147
1248
  checksum = new RouteKeysChecksum();
1148
1249
  batch = [];
@@ -1155,6 +1256,7 @@ class RouteStreamSender {
1155
1256
  this.control = control;
1156
1257
  this.options = options;
1157
1258
  this.streamId = options.streamId;
1259
+ this.telemetry = options.telemetry ?? new SdkPerformanceCollector();
1158
1260
  }
1159
1261
  async begin() {
1160
1262
  await this.control.request("transfer.route_stream.begin", compact({
@@ -1216,7 +1318,11 @@ class RouteStreamSender {
1216
1318
  await this.sendTail;
1217
1319
  if (this.sendError)
1218
1320
  throw this.sendError;
1321
+ this.telemetry.observe("sdk.preparation", performance.now() - this.started);
1322
+ const sdkPerformance = this.telemetry.snapshot();
1323
+ this.options.onDiagnostics?.(sdkPerformance);
1219
1324
  return this.control.request("transfer.route_stream.complete", {
1325
+ sdk_performance: sdkPerformance,
1220
1326
  transfer_id: this.options.transferId,
1221
1327
  route_generation_id: this.options.routeGenerationId,
1222
1328
  stream_id: this.streamId,
@@ -1231,7 +1337,7 @@ class RouteStreamSender {
1231
1337
  async enqueueFlush() {
1232
1338
  if (!this.batch.length)
1233
1339
  return;
1234
- await this.sendTail;
1340
+ await this.telemetry.measure("sdk.buffer_wait", () => this.sendTail);
1235
1341
  if (this.sendError)
1236
1342
  throw this.sendError;
1237
1343
  const routes = this.batch.splice(0, this.batch.length);
@@ -1255,7 +1361,8 @@ class RouteStreamSender {
1255
1361
  const routeChecksum = chunkChecksum.value();
1256
1362
  const coordinateChecksum = await routeCoordinateChecksum(chunk);
1257
1363
  const batchId = `${this.streamId}:${batchIndex}:${coordinateChecksum}`;
1258
- await this.control.request("transfer.route_stream.batch", {
1364
+ this.telemetry.counters.route_batches++;
1365
+ await this.telemetry.measure("sdk.batch_ack", () => this.control.request("transfer.route_stream.batch", {
1259
1366
  transfer_id: this.options.transferId,
1260
1367
  route_generation_id: this.options.routeGenerationId,
1261
1368
  stream_id: this.streamId,
@@ -1267,7 +1374,7 @@ class RouteStreamSender {
1267
1374
  }, {
1268
1375
  transferId: this.options.transferId,
1269
1376
  idempotencyKey: `transfer:${this.options.transferId}:route-stream:${this.streamId}:batch:${batchIndex}:${coordinateChecksum}`
1270
- });
1377
+ }));
1271
1378
  }
1272
1379
  catch (error) {
1273
1380
  this.sendError = error;
@@ -1511,6 +1618,9 @@ function createDeferred() {
1511
1618
  resolve = resolvePromise;
1512
1619
  reject = rejectPromise;
1513
1620
  });
1621
+ // A waiter may be rejected after its route loop already stopped (for example when
1622
+ // ownership is fenced mid-stream); awaiting callers still observe the rejection.
1623
+ promise.catch(() => undefined);
1514
1624
  return { promise, resolve, reject };
1515
1625
  }
1516
1626
  function createMultipartGroupWaiters(prepared, destinationsById) {
package/dist/models.d.ts CHANGED
@@ -1,5 +1,21 @@
1
1
  export type SignedUrlFlow = "signed_url";
2
+ export interface SdkPerformanceSummary {
3
+ schema_version: "sdk-performance/v1";
4
+ measurements: Array<{
5
+ name: "sdk.discovery" | "sdk.multipart_create" | "sdk.signing" | "sdk.batch_ack" | "sdk.buffer_wait" | "sdk.preparation";
6
+ count: number;
7
+ work_ms: number;
8
+ max_ms: number;
9
+ }>;
10
+ counters: {
11
+ source_signatures: number;
12
+ source_reuses: number;
13
+ route_batches: number;
14
+ };
15
+ }
2
16
  export interface BeamClientOptions {
17
+ /** Best effort, at most one bounded summary per route stream. Never awaited. */
18
+ onDiagnostics?: (summary: SdkPerformanceSummary) => void | Promise<void>;
3
19
  apiKey?: string;
4
20
  natsUrl?: string;
5
21
  natsWsUrl?: string;
@@ -141,7 +157,34 @@ export interface IntegrityAuditChallenge {
141
157
  range_bytes: number;
142
158
  chunks: IntegrityAuditChallengeChunk[];
143
159
  }
160
+ export interface TransferPerformance {
161
+ schema_version: "transfer-performance/v1";
162
+ runtime_epoch: string;
163
+ coverage: "complete" | "restarted";
164
+ elapsed_ms: number;
165
+ /** Work can overlap; these durations must not be summed into elapsed time. */
166
+ measurements: Record<string, {
167
+ count: number;
168
+ work_ms: number;
169
+ max_ms: number;
170
+ }>;
171
+ events: Record<string, {
172
+ count: number;
173
+ first_ms: number;
174
+ last_ms: number;
175
+ }>;
176
+ sdk_counters: {
177
+ source_signatures: number;
178
+ source_reuses: number;
179
+ route_batches: number;
180
+ };
181
+ detail_dropped: number;
182
+ unmeasured?: string[];
183
+ measurement_clocks?: Record<string, "runtime" | "sdk" | "fraud-service">;
184
+ sdk_report_received?: boolean;
185
+ }
144
186
  export interface TransferStatusInfo {
187
+ performance?: TransferPerformance | null;
145
188
  transfer_id: string;
146
189
  name?: string;
147
190
  status: string;
@@ -1,3 +1,4 @@
1
+ import type { IntegrityAuditChallenge } from "./models.js";
1
2
  import { type ConnectionOptions } from "nats";
2
3
  import type { MultipartGroupManifest, SignedChunkRoute, TransferTerminalSignalWaiter } from "./models.js";
3
4
  export declare const TRANSFER_CLIENT_CONTROL_SCHEMA_VERSION = "transfer-client-control/v7";
@@ -137,10 +138,18 @@ export declare class BeamTransferControl {
137
138
  private helloTimer;
138
139
  constructor(options: TransferControlOptions);
139
140
  close(): Promise<void>;
141
+ serveIntegritySigner(transferId: string, handler: (challenge: IntegrityAuditChallenge) => Promise<Record<string, unknown>>): Promise<() => void>;
140
142
  serveRouteRecoverySigner(transferId: string, handler: (payload: RouteRecoverySignRequestPayload) => Promise<RouteRecoverySignReplyPayload>): Promise<() => void>;
141
143
  registerRecoveryLease(lease: TransferRecoveryLease): () => void;
142
- releaseRecoveryLease(transferId: string): void;
143
- continueRecoveryLease(transferId: string): void;
144
+ /**
145
+ * Release a transfer's recovery lease, disposing its retained secrets.
146
+ *
147
+ * With `owner`, release only when that exact lease is still registered, so a
148
+ * fenced-off owner can never release a replacement owner's lease.
149
+ */
150
+ releaseRecoveryLease(transferId: string, owner?: TransferRecoveryLease): void;
151
+ /** Request recovery for the registered lease; with `owner`, only if that lease is still registered. */
152
+ continueRecoveryLease(transferId: string, owner?: TransferRecoveryLease): void;
144
153
  request<T>(messageType: TransferClientMessageType, payload: Record<string, unknown>, options?: LifecycleRequestOptions): Promise<T>;
145
154
  openTerminalSignalWaiter(transferId: string): Promise<TransferTerminalSignalWaiter>;
146
155
  splitRoutesForPayload(messageType: TransferClientMessageType, basePayload: Record<string, unknown>, routes: SignedChunkRoute[]): SignedChunkRoute[][];
@@ -60,6 +60,8 @@ export class BeamTransferControl {
60
60
  terminalWaitCancellations = new Set();
61
61
  routeRecoverySignerCancellations = new Set();
62
62
  recoveryLeases = new Map();
63
+ // Keyed by lease identity: a replacement owner's recovery must never be
64
+ // coalesced into (and silently dropped behind) a fenced-off owner's loop.
63
65
  recoveryOperations = new Map();
64
66
  recoveryRequestedEpoch = new Map();
65
67
  runtimeEpochs = new Map();
@@ -106,6 +108,58 @@ export class BeamTransferControl {
106
108
  if (nc && !nc.isClosed())
107
109
  await nc.drain();
108
110
  }
111
+ async serveIntegritySigner(transferId, handler) {
112
+ const nc = await this.connection();
113
+ const subject = this.routeRecoverySignSubject(transferId).replace(/route_recovery_sign$/, "integrity_sign");
114
+ let active = true;
115
+ let pending = null;
116
+ let auditId = "";
117
+ let challengeFingerprint = "";
118
+ let awaiting = 0;
119
+ const subscription = nc.subscribe(subject, { callback: (error, message) => {
120
+ if (!active || error || !message.reply)
121
+ return;
122
+ if (awaiting >= 32) {
123
+ message.respond(new TextEncoder().encode('{"retry":true}'));
124
+ return;
125
+ }
126
+ awaiting++;
127
+ void (async () => {
128
+ try {
129
+ const request = JSON.parse(new TextDecoder().decode(message.data));
130
+ if (request.schema_version !== TRANSFER_CLIENT_CONTROL_SCHEMA_VERSION || request.capability !== "integrity-signing/v1"
131
+ || request.producer !== "transfer-runtime" || request.environment !== this.environment || request.key_prefix !== this.keyPrefix
132
+ || request.transfer_id !== transferId || request.challenge?.transfer_id !== transferId || typeof request.request_id !== "string")
133
+ throw new Error("integrity_sign_scope_mismatch");
134
+ const fingerprint = JSON.stringify(request.challenge);
135
+ if (pending && (auditId !== request.challenge.audit_id || challengeFingerprint !== fingerprint))
136
+ throw new Error("integrity_sign_conflicting_audit");
137
+ if (!pending) {
138
+ auditId = request.challenge.audit_id;
139
+ challengeFingerprint = fingerprint;
140
+ pending = handler(request.challenge).finally(() => { pending = null; });
141
+ }
142
+ const payload = await pending;
143
+ if (active)
144
+ message.respond(new TextEncoder().encode(JSON.stringify({ schema_version: TRANSFER_CLIENT_CONTROL_SCHEMA_VERSION,
145
+ capability: "integrity-signing/v1", environment: this.environment, key_prefix: this.keyPrefix,
146
+ transfer_id: transferId, request_id: request.request_id, payload })));
147
+ }
148
+ catch {
149
+ message.respond(new TextEncoder().encode('{"retry":true}'));
150
+ }
151
+ finally {
152
+ awaiting--;
153
+ }
154
+ })();
155
+ } });
156
+ await nc.flush();
157
+ const stop = () => { if (!active)
158
+ return; active = false; subscription.unsubscribe(); this.routeRecoverySignerCancellations.delete(stop); };
159
+ this.routeRecoverySignerCancellations.add(stop);
160
+ void this.pollRuntimeHello(transferShardId(transferId, this.shardCount));
161
+ return stop;
162
+ }
109
163
  async serveRouteRecoverySigner(transferId, handler) {
110
164
  const nc = await this.connection();
111
165
  const subject = this.routeRecoverySignSubject(transferId);
@@ -165,23 +219,20 @@ export class BeamTransferControl {
165
219
  this.ensureHelloMonitor();
166
220
  const shardId = transferShardId(lease.transferId, this.shardCount);
167
221
  void this.pollRuntimeHello(shardId);
168
- return () => {
169
- const current = this.recoveryLeases.get(lease.transferId);
170
- if (current !== lease)
171
- return;
172
- this.recoveryLeases.delete(lease.transferId);
173
- this.recoveryRequestedEpoch.delete(lease.transferId);
174
- lease.disposeSecrets?.();
175
- if (!this.recoveryLeases.size && this.helloTimer) {
176
- clearInterval(this.helloTimer);
177
- this.helloTimer = null;
178
- }
179
- };
222
+ return () => this.releaseRecoveryLease(lease.transferId, lease);
180
223
  }
181
- releaseRecoveryLease(transferId) {
224
+ /**
225
+ * Release a transfer's recovery lease, disposing its retained secrets.
226
+ *
227
+ * With `owner`, release only when that exact lease is still registered, so a
228
+ * fenced-off owner can never release a replacement owner's lease.
229
+ */
230
+ releaseRecoveryLease(transferId, owner) {
182
231
  const lease = this.recoveryLeases.get(transferId);
183
232
  if (!lease)
184
233
  return;
234
+ if (owner && lease !== owner)
235
+ return;
185
236
  this.recoveryLeases.delete(transferId);
186
237
  this.recoveryRequestedEpoch.delete(transferId);
187
238
  lease.disposeSecrets?.();
@@ -190,10 +241,13 @@ export class BeamTransferControl {
190
241
  this.helloTimer = null;
191
242
  }
192
243
  }
193
- continueRecoveryLease(transferId) {
244
+ /** Request recovery for the registered lease; with `owner`, only if that lease is still registered. */
245
+ continueRecoveryLease(transferId, owner) {
194
246
  const lease = this.recoveryLeases.get(transferId);
195
247
  if (!lease || this.closed)
196
248
  return;
249
+ if (owner && lease !== owner)
250
+ return;
197
251
  const shardId = transferShardId(transferId, this.shardCount);
198
252
  const epoch = this.runtimeEpochs.get(shardId);
199
253
  this.recoveryRequestedEpoch.set(transferId, epoch ? `${epoch.runtimeEpoch}:${epoch.transportEpoch}` : `foreground:${randomId()}`);
@@ -436,7 +490,12 @@ export class BeamTransferControl {
436
490
  }
437
491
  async pollRuntimeHello(shardId) {
438
492
  try {
439
- await this.request("runtime.hello", {}, { idempotencyKey: `runtime:hello:${shardId}`, shardId });
493
+ try {
494
+ await this.request("runtime.hello", { capabilities: ["integrity-signing/v1"] }, { idempotencyKey: `runtime:hello:${shardId}:integrity-signing`, shardId });
495
+ }
496
+ catch {
497
+ await this.request("runtime.hello", {}, { idempotencyKey: `runtime:hello:${shardId}`, shardId });
498
+ }
440
499
  }
441
500
  catch {
442
501
  // Infinite NATS reconnect continues in the background; the next hello reconciles epochs.
@@ -463,13 +522,14 @@ export class BeamTransferControl {
463
522
  }
464
523
  }
465
524
  recoverLease(lease) {
466
- const existing = this.recoveryOperations.get(lease.transferId);
525
+ const existing = this.recoveryOperations.get(lease);
467
526
  if (existing)
468
527
  return existing;
469
- let operation;
470
- operation = (async () => {
528
+ const owns = () => !this.closed && this.recoveryLeases.get(lease.transferId) === lease;
529
+ let replaced = false;
530
+ const operation = (async () => {
471
531
  let attempt = 0;
472
- while (!this.closed && this.recoveryLeases.get(lease.transferId) === lease) {
532
+ while (owns()) {
473
533
  const requestedEpoch = this.recoveryRequestedEpoch.get(lease.transferId);
474
534
  const routeGenerationId = randomId();
475
535
  try {
@@ -479,39 +539,53 @@ export class BeamTransferControl {
479
539
  coordinate_checksum: lease.coordinateChecksum,
480
540
  route_generation_id: routeGenerationId
481
541
  }, { transferId: lease.transferId, idempotencyKey: `transfer:${lease.transferId}:resume:${routeGenerationId}` });
542
+ if (!owns()) {
543
+ // A fenced-off owner must neither act on nor release a replacement's lease.
544
+ replaced = true;
545
+ return;
546
+ }
482
547
  if (response.recovery === "terminal") {
483
- this.releaseRecoveryLease(lease.transferId);
484
- if (this.recoveryOperations.get(lease.transferId) === operation) {
485
- this.recoveryOperations.delete(lease.transferId);
486
- }
548
+ this.releaseRecoveryLease(lease.transferId, lease);
487
549
  return;
488
550
  }
489
551
  if (response.route_replay_required || response.recovery === "route_replay_required") {
490
552
  await lease.replayRoutes(routeGenerationId);
553
+ if (!owns()) {
554
+ replaced = true;
555
+ return;
556
+ }
491
557
  }
492
558
  if (this.recoveryRequestedEpoch.get(lease.transferId) !== requestedEpoch) {
493
559
  attempt = 0;
494
560
  continue;
495
561
  }
496
- if (this.recoveryOperations.get(lease.transferId) === operation) {
497
- this.recoveryOperations.delete(lease.transferId);
498
- }
499
562
  return;
500
563
  }
501
564
  catch (error) {
565
+ if (!owns()) {
566
+ replaced = true;
567
+ return;
568
+ }
502
569
  if (!isRetryableLifecycleError(error)) {
503
- this.releaseRecoveryLease(lease.transferId);
570
+ this.releaseRecoveryLease(lease.transferId, lease);
504
571
  return;
505
572
  }
506
573
  attempt += 1;
507
574
  await sleepWithJitter(Math.min(30_000, 500 * (2 ** Math.min(attempt, 6))));
508
575
  }
509
576
  }
577
+ replaced = !this.closed;
510
578
  })().finally(() => {
511
- if (this.recoveryOperations.get(lease.transferId) === operation)
512
- this.recoveryOperations.delete(lease.transferId);
579
+ if (this.recoveryOperations.get(lease) === operation)
580
+ this.recoveryOperations.delete(lease);
581
+ if (!replaced || this.closed)
582
+ return;
583
+ // The recovery this loop was driving still matters: hand it to the replacement owner.
584
+ const current = this.recoveryLeases.get(lease.transferId);
585
+ if (current && current !== lease)
586
+ void this.recoverLease(current);
513
587
  });
514
- this.recoveryOperations.set(lease.transferId, operation);
588
+ this.recoveryOperations.set(lease, operation);
515
589
  return operation;
516
590
  }
517
591
  async resolveAuthToken() {
@@ -0,0 +1,12 @@
1
+ import type { SdkPerformanceSummary } from './models.js';
2
+ export declare class SdkPerformanceCollector {
3
+ private readonly measurements;
4
+ readonly counters: {
5
+ source_signatures: number;
6
+ source_reuses: number;
7
+ route_batches: number;
8
+ };
9
+ observe(name: SdkPerformanceSummary['measurements'][number]['name'], milliseconds: number): void;
10
+ measure<T>(name: SdkPerformanceSummary['measurements'][number]['name'], work: () => Promise<T>): Promise<T>;
11
+ snapshot(): SdkPerformanceSummary;
12
+ }
@@ -0,0 +1,25 @@
1
+ export class SdkPerformanceCollector {
2
+ measurements = new Map();
3
+ counters = { source_signatures: 0, source_reuses: 0, route_batches: 0 };
4
+ observe(name, milliseconds) {
5
+ if (!Number.isFinite(milliseconds) || milliseconds < 0)
6
+ return;
7
+ const value = this.measurements.get(name) ?? { count: 0, work_ms: 0, max_ms: 0 };
8
+ value.count++;
9
+ value.work_ms += milliseconds;
10
+ value.max_ms = Math.max(value.max_ms, milliseconds);
11
+ this.measurements.set(name, value);
12
+ }
13
+ async measure(name, work) {
14
+ const start = performance.now();
15
+ try {
16
+ return await work();
17
+ }
18
+ finally {
19
+ this.observe(name, performance.now() - start);
20
+ }
21
+ }
22
+ snapshot() {
23
+ return { schema_version: 'sdk-performance/v1', measurements: [...this.measurements].map(([name, value]) => ({ name: name, ...value })), counters: { ...this.counters } };
24
+ }
25
+ }
@@ -63,7 +63,15 @@ export declare function inspectDestinationObject(destination: ProviderDestinatio
63
63
  versionId: string | undefined;
64
64
  metadata: Record<string, string>;
65
65
  }>;
66
+ export declare function boundedGrantExpiry(url: string, upperBound: string): string;
67
+ export type SourceRouteGrant = {
68
+ url: string;
69
+ headers: Record<string, string>;
70
+ expiresAt: string;
71
+ };
72
+ export declare function signSourceChunk(input: Parameters<typeof signSourceRoute>[0]): Promise<SourceRouteGrant>;
66
73
  export declare function signDestinationRoute(input: {
74
+ sourceGrant?: Promise<SourceRouteGrant>;
67
75
  chunk: ChunkSigningPlanItem;
68
76
  target: ChunkDestinationSigningTarget;
69
77
  source?: ProviderSourceConfig;
@@ -109,6 +117,16 @@ export declare function signDestinationReadRange(input: {
109
117
  url: string;
110
118
  headers: Record<string, string>;
111
119
  }>;
120
+ declare function signSourceRoute(input: {
121
+ source?: ProviderSourceConfig;
122
+ fallbackUrl: string;
123
+ chunk: ChunkSigningPlanItem;
124
+ expiresIn: number;
125
+ fetchImpl?: typeof fetch;
126
+ }): Promise<{
127
+ url: string;
128
+ headers: Record<string, string>;
129
+ }>;
112
130
  export declare function signDestinationUrl(input: {
113
131
  contentMd5?: string;
114
132
  destination: ProviderDestinationConfig;
@@ -118,8 +136,10 @@ export declare function signDestinationUrl(input: {
118
136
  expiresIn: number;
119
137
  fetchImpl?: typeof fetch;
120
138
  }): Promise<string>;
139
+ export declare function releaseProviderClients(configs: Array<ProviderSourceConfig | ProviderDestinationConfig>): void;
121
140
  export declare function isS3CompatibleProvider(config: ProviderSourceConfig | ProviderDestinationConfig): config is AnyS3CompatibleProviderConfig;
122
141
  export declare function s3CompatibleEndpoint(source: AnyS3CompatibleProviderConfig): string | undefined;
123
142
  export declare function s3CompatibleRegion(source: AnyS3CompatibleProviderConfig): string;
124
143
  export declare function s3CompatibleForcePathStyle(source: AnyS3CompatibleProviderConfig, endpoint?: string | undefined): boolean | undefined;
125
144
  export declare function isHuggingFaceProvider(config: ProviderSourceConfig | ProviderDestinationConfig): config is HuggingFaceProviderConfig;
145
+ export {};
@@ -69,7 +69,7 @@ export function nowIso() {
69
69
  return new Date().toISOString();
70
70
  }
71
71
  export function expiresAtIso(expiresIn) {
72
- return new Date(Date.now() + expiresIn * 1000).toISOString();
72
+ return new Date((Math.floor(Date.now() / 1000) + expiresIn) * 1000).toISOString();
73
73
  }
74
74
  export async function prepareProviderSource(source, options = {}) {
75
75
  const index = options.index ?? 0;
@@ -307,11 +307,34 @@ export async function inspectDestinationObject(destination, objectKey, signal) {
307
307
  const result = await createS3CompatibleClient(destination).send(new HeadObjectCommand({ Bucket: destination.bucket, Key: objectKey }), { abortSignal: signal });
308
308
  return { size: result.ContentLength, etag: result.ETag, versionId: result.VersionId, metadata: result.Metadata ?? {} };
309
309
  }
310
+ // Never advertise a renewed lifetime for a previously issued provider URL.
311
+ export function boundedGrantExpiry(url, upperBound) {
312
+ let expiry = Date.parse(upperBound);
313
+ const query = new URL(url).searchParams;
314
+ for (const prefix of ["X-Amz", "X-Goog"]) {
315
+ const date = query.get(`${prefix}-Date`), duration = Number(query.get(`${prefix}-Expires`));
316
+ if (date && /^\d{8}T\d{6}Z$/.test(date) && Number.isFinite(duration) && duration > 0) {
317
+ const start = Date.parse(`${date.slice(0, 4)}-${date.slice(4, 6)}-${date.slice(6, 8)}T${date.slice(9, 11)}:${date.slice(11, 13)}:${date.slice(13, 15)}Z`);
318
+ if (Number.isFinite(start))
319
+ expiry = Math.min(expiry, start + duration * 1000);
320
+ }
321
+ }
322
+ const epoch = Number(query.get("Expires"));
323
+ if (Number.isFinite(epoch) && epoch > 0)
324
+ expiry = Math.min(expiry, epoch * 1000);
325
+ return new Date(expiry).toISOString();
326
+ }
327
+ export async function signSourceChunk(input) {
328
+ const expiresAt = expiresAtIso(input.expiresIn);
329
+ const grant = await signSourceRoute(input);
330
+ return { ...grant, expiresAt: boundedGrantExpiry(grant.url, expiresAt) };
331
+ }
310
332
  export async function signDestinationRoute(input) {
311
333
  const targetObjectKey = input.target.object_key;
312
334
  if (!targetObjectKey) {
313
335
  throw new Error("destination signing target is missing object_key");
314
336
  }
337
+ const destinationExpiresAt = expiresAtIso(input.expiresIn);
315
338
  const [destUrl, sourceRoute] = await Promise.all([
316
339
  input.destUrl ?? signDestinationUrl({
317
340
  destination: input.destination,
@@ -321,7 +344,7 @@ export async function signDestinationRoute(input) {
321
344
  expiresIn: input.expiresIn,
322
345
  fetchImpl: input.fetchImpl,
323
346
  }),
324
- signSourceRoute({
347
+ input.sourceGrant ?? signSourceChunk({
325
348
  source: input.source,
326
349
  fallbackUrl: input.chunk.source_url,
327
350
  chunk: input.chunk,
@@ -337,8 +360,8 @@ export async function signDestinationRoute(input) {
337
360
  dest_url: destUrl,
338
361
  source_offset: input.chunk.source_offset,
339
362
  chunk_size: input.chunk.chunk_size,
340
- expires_at: expiresAtIso(input.expiresIn),
341
- headers: sourceRoute.headers,
363
+ expires_at: boundedGrantExpiry(destUrl, new Date(Math.min(Date.parse(destinationExpiresAt), Date.parse(sourceRoute.expiresAt))).toISOString()),
364
+ headers: { ...sourceRoute.headers },
342
365
  metadata: {
343
366
  ...(input.target.metadata ?? {}),
344
367
  ...(input.multipartGroupId ? { multipart_group_id: input.multipartGroupId } : {}),
@@ -475,7 +498,16 @@ export async function signDestinationUrl(input) {
475
498
  }
476
499
  return unsupportedProviderConfig(input.destination);
477
500
  }
478
- const s3ClientCache = new Map();
501
+ const s3ClientCache = new WeakMap();
502
+ export function releaseProviderClients(configs) {
503
+ for (const config of configs) {
504
+ if (!isS3CompatibleProvider(config))
505
+ continue;
506
+ const cached = s3ClientCache.get(config);
507
+ s3ClientCache.delete(config);
508
+ cached?.client.destroy();
509
+ }
510
+ }
479
511
  function s3ClientCacheKey(source, endpoint) {
480
512
  return JSON.stringify({
481
513
  provider: source.provider,
@@ -489,9 +521,9 @@ function s3ClientCacheKey(source, endpoint) {
489
521
  }
490
522
  function createS3CompatibleClient(source, endpoint = s3CompatibleEndpoint(source)) {
491
523
  const cacheKey = s3ClientCacheKey(source, endpoint);
492
- const cached = s3ClientCache.get(cacheKey);
493
- if (cached)
494
- return cached;
524
+ const cached = s3ClientCache.get(source);
525
+ if (cached?.identity === cacheKey)
526
+ return cached.client;
495
527
  const client = new S3Client({
496
528
  region: s3CompatibleRegion(source),
497
529
  endpoint,
@@ -503,7 +535,7 @@ function createS3CompatibleClient(source, endpoint = s3CompatibleEndpoint(source
503
535
  sessionToken: "session_token" in source ? source.session_token : undefined
504
536
  }
505
537
  });
506
- s3ClientCache.set(cacheKey, client);
538
+ s3ClientCache.set(source, { identity: cacheKey, client });
507
539
  return client;
508
540
  }
509
541
  export function isS3CompatibleProvider(config) {
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@beam-network/sdk",
3
- "version": "0.6.0",
3
+ "version": "0.6.1",
4
4
  "description": "TypeScript SDK for BEAM transfer creation and management.",
5
5
  "type": "module",
6
6
  "license": "MIT",