@fluidframework/container-runtime 2.0.0-dev.4.1.0.148229 → 2.0.0-dev.4.2.0.153917

package/CHANGELOG.md

@@ -0,0 +1,58 @@
+# @fluidframework/container-runtime
+
+## 2.0.0-internal.4.1.0
+
+### Minor Changes
+
+-   GC interfaces removed from runtime-definitions ([#14750](https://github.com/microsoft/FluidFramework/pull-requests/14750)) [60274eacab](https://github.com/microsoft/FluidFramework/commits/60274eacabf14d42f52f6ad1c2f64356e64ba1a2)
+
+    The following interfaces available in `@fluidframework/runtime-definitions` are internal implementation details and have been deprecated for public use. They will be removed in an upcoming release.
+
+    -   `IGarbageCollectionNodeData`
+    -   `IGarbageCollectionState`
+    -   `IGarbageCollectionSnapshotData`
+    -   `IGarbageCollectionSummaryDetailsLegacy`
+
+-   Ability to enable grouped batching ([#14512](https://github.com/microsoft/FluidFramework/pull-requests/14512)) [8e4dc47a38](https://github.com/microsoft/FluidFramework/commits/8e4dc47a3871bcaf1f7c1339c362d9c9d08551fc)
+
+    The `IContainerRuntimeOptions.enableGroupedBatching` option has been added to the container runtime layer and is off by default. This option will group all batch messages
+    under a new "grouped" message to be sent to the service. Upon receiving this new "grouped" message, the batch messages will be extracted and given
+    the sequence number of the parent "grouped" message.
+
+    Upon enabling this option, if any issues arise, use the `Fluid.ContainerRuntime.DisableGroupedBatching` feature flag to disable at runtime. This option should **ONLY** be enabled after observing that 99.9% of your application sessions contains these changes (version "2.0.0-internal.4.1.0" or later). This option is experimental and should not be enabled yet in production. Containers created with this option may not open in future versions of the framework.
+
+    This option will change a couple of expectations around message structure and runtime layer expectations. Only enable this option after testing
+    and verifying that the following expectation changes won't have any effects:
+
+    -   batch messages observed at the runtime layer will not match messages seen at the loader layer
+    -   messages within the same batch will have the same sequence number
+    -   client sequence numbers on batch messages can only be used to order messages with the same sequenceNumber
+    -   requires all ops to be processed by runtime layer (version "2.0.0-internal.1.2.0" or later
+        https://github.com/microsoft/FluidFramework/pull/11832)
+
+-   Op compression is enabled by default ([#14856](https://github.com/microsoft/FluidFramework/pull-requests/14856)) [439c21f31f](https://github.com/microsoft/FluidFramework/commits/439c21f31f4a3ea6515f01d2b2be7f35c04910ce)
+
+    If the size of a batch is larger than 614kb, the ops will be compressed. After upgrading to this version, if batches exceed the size threshold, the runtime will produce a new type of op with the compression properties. To open a document which contains this type of op, the client's runtime version needs to be at least `client_v2.0.0-internal.2.3.0`. Older clients will close with assert `0x3ce` ("Runtime message of unknown type") and will not be able to open the documents until they upgrade. To minimize the risk, it is recommended to audit existing session and ensure that at least 99.9% of them are using a runtime version equal or greater than `client_v2.0.0-internal.2.3.0`, before upgrading to `2.0.0-internal.4.1.0`.
+
+    More information about op compression can be found
+    [here](./packages/runtime/container-runtime/src/opLifecycle/README.md).
+
+-   @fluidframework/garbage-collector deprecated ([#14750](https://github.com/microsoft/FluidFramework/pull-requests/14750)) [60274eacab](https://github.com/microsoft/FluidFramework/commits/60274eacabf14d42f52f6ad1c2f64356e64ba1a2)
+
+    The `@fluidframework/garbage-collector` package is deprecated with the following functions, interfaces, and types in it.
+    These are internal implementation details and have been deprecated for public use. They will be removed in an upcoming
+    release.
+
+    -   `cloneGCData`
+    -   `concatGarbageCollectionData`
+    -   `concatGarbageCollectionStates`
+    -   `GCDataBuilder`
+    -   `getGCDataFromSnapshot`
+    -   `IGCResult`
+    -   `removeRouteFromAllNodes`
+    -   `runGarbageCollection`
+    -   `trimLeadingAndTrailingSlashes`
+    -   `trimLeadingSlashes`
+    -   `trimTrailingSlashes`
+    -   `unpackChildNodesGCDetails`
+    -   `unpackChildNodesUsedRoutes`

package/README.md

@@ -1 +1,70 @@
 # @fluidframework/container-runtime
+
+<!-- AUTO-GENERATED-CONTENT:START (LIBRARY_PACKAGE_README:scripts=FALSE) -->
+
+<!-- prettier-ignore-start -->
+<!-- NOTE: This section is automatically generated using @fluid-tools/markdown-magic. Do not update these generated contents directly. -->
+
+## Using Fluid Framework libraries
+
+When taking a dependency on a Fluid Framework library, we recommend using a `^` (caret) version range, such as `^1.3.4`.
+While Fluid Framework libraries may use different ranges with interdependencies between other Fluid Framework libraries,
+library consumers should always prefer `^`.
+
+Note that when depending on a library version of the form 2.0.0-internal.x.y.z, called the Fluid internal version
+scheme, you must use a `>= <` dependency range. Standard `^` and `~` ranges will not work as expected. See the
+[@fluid-tools/version-tools](https://github.com/microsoft/FluidFramework/blob/main/build-tools/packages/version-tools/README.md)
+package for more information including tools to convert between version schemes.
+
+## Installation
+
+To get started, install the package by running the following command:
+
+```bash
+npm i @fluidframework/container-runtime
+```
+
+## API Documentation
+
+API documentation for **@fluidframework/container-runtime** is available at <https://fluidframework.com/docs/apis/container-runtime>.
+
+## Contribution Guidelines
+
+There are many ways to [contribute](https://github.com/microsoft/FluidFramework/blob/main/CONTRIBUTING.md) to Fluid.
+
+-   Participate in Q&A in our [GitHub Discussions](https://github.com/microsoft/FluidFramework/discussions).
+-   [Submit bugs](https://github.com/microsoft/FluidFramework/issues) and help us verify fixes as they are checked in.
+-   Review the [source code changes](https://github.com/microsoft/FluidFramework/pulls).
+-   [Contribute bug fixes](https://github.com/microsoft/FluidFramework/blob/main/CONTRIBUTING.md).
+
+Detailed instructions for working in the repo can be found in the [Wiki](https://github.com/microsoft/FluidFramework/wiki).
+
+This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/).
+For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments.
+
+This project may contain Microsoft trademarks or logos for Microsoft projects, products, or services.
+Use of these trademarks or logos must follow Microsoft’s [Trademark & Brand Guidelines](https://www.microsoft.com/trademarks).
+Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship.
+
+## Help
+
+Not finding what you're looking for in this README? Check out our [GitHub
+Wiki](https://github.com/microsoft/FluidFramework/wiki) or [fluidframework.com](https://fluidframework.com/docs/).
+
+Still not finding what you're looking for? Please [file an
+issue](https://github.com/microsoft/FluidFramework/wiki/Submitting-Bugs-and-Feature-Requests).
+
+Thank you!
+
+## Trademark
+
+This project may contain Microsoft trademarks or logos for Microsoft projects, products, or services.
+
+Use of these trademarks or logos must follow Microsoft's [Trademark & Brand
+Guidelines](https://www.microsoft.com/en-us/legal/intellectualproperty/trademarks/usage/general).
+
+Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship.
+
+<!-- prettier-ignore-end -->
+
+<!-- AUTO-GENERATED-CONTENT:END -->

package/dist/blobManager.d.ts

@@ -7,6 +7,7 @@ import { IDocumentStorageService } from "@fluidframework/driver-definitions";
 import { ISequencedDocumentMessage, ISnapshotTree } from "@fluidframework/protocol-definitions";
 import { TypedEventEmitter } from "@fluidframework/common-utils";
 import { IContainerRuntime, IContainerRuntimeEvents } from "@fluidframework/container-runtime-definitions";
+import { ICriticalContainerError } from "@fluidframework/container-definitions";
 import { IGarbageCollectionData, ISummaryTreeWithStats, ITelemetryContext } from "@fluidframework/runtime-definitions";
 import { ContainerRuntime } from "./containerRuntime";
 /**
@@ -39,6 +40,8 @@ export declare type IBlobManagerRuntime = Pick<IContainerRuntime, "attachState"
 export interface IPendingBlobs {
     [id: string]: {
         blob: string;
+        uploadTime?: number;
+        minTTLInSeconds?: number;
     };
 }
 export interface IBlobManagerEvents {
@@ -47,21 +50,10 @@ export interface IBlobManagerEvents {
 export declare class BlobManager extends TypedEventEmitter<IBlobManagerEvents> {
     private readonly routeContext;
     private readonly getStorage;
-    /**
-     * Submit a BlobAttach op. When a blob is uploaded, there is a short grace period before which the blob is
-     * deleted. The BlobAttach op notifies the server that blob is in use. The server will then not delete the
-     * the blob as long as it is listed as referenced in future summaries. The summarizing client will know to
-     * include the storage ID in the summary when it sees the op.
-     *
-     * The op will also include a local ID to inform all clients of the relation to the storage ID, without
-     * knowledge of which they cannot request the blob from storage. It's important that this op is sequenced
-     * before any ops that reference the local ID, otherwise, an invalid handle could be added to the document.
-     */
-    private readonly sendBlobAttachOp;
     private readonly blobRequested;
     private readonly isBlobDeleted;
     private readonly runtime;
-    private readonly getCurrentReferenceTimestampMs;
+    private readonly closeContainer;
     static readonly basePath = "_blobs";
     private static readonly redirectTableBlobName;
     private readonly mc;
@@ -93,6 +85,7 @@ export declare class BlobManager extends TypedEventEmitter<IBlobManagerEvents> {
      * Tombstone is a temporary feature that imitates a blob getting swept by garbage collection.
      */
     private readonly tombstonedBlobs;
+    private readonly sendBlobAttachOp;
     constructor(routeContext: IFluidHandleContext, snapshot: IBlobManagerLoadInfo, getStorage: () => IDocumentStorageService, 
     /**
      * Submit a BlobAttach op. When a blob is uploaded, there is a short grace period before which the blob is
@@ -104,7 +97,7 @@ export declare class BlobManager extends TypedEventEmitter<IBlobManagerEvents> {
      * knowledge of which they cannot request the blob from storage. It's important that this op is sequenced
      * before any ops that reference the local ID, otherwise, an invalid handle could be added to the document.
      */
-    sendBlobAttachOp: (localId: string, storageId?: string) => void, blobRequested: (blobPath: string) => void, isBlobDeleted: (blobPath: string) => boolean, runtime: IBlobManagerRuntime, stashedBlobs: IPendingBlobs | undefined, getCurrentReferenceTimestampMs: () => number | undefined);
+    sendBlobAttachOp: (localId: string, storageId?: string) => void, blobRequested: (blobPath: string) => void, isBlobDeleted: (blobPath: string) => boolean, runtime: IBlobManagerRuntime, stashedBlobs: IPendingBlobs | undefined, closeContainer: (error?: ICriticalContainerError) => void);
     private get pendingOfflineUploads();
     get hasPendingOfflineUploads(): boolean;
     get hasPendingBlobs(): boolean;
@@ -142,7 +135,6 @@ export declare class BlobManager extends TypedEventEmitter<IBlobManagerEvents> {
      * @param metadata - op metadata containing storage and/or local IDs
      */
     reSubmit(metadata: Record<string, unknown> | undefined): void;
-    private logTimeInfo;
     processBlobAttachOp(message: ISequencedDocumentMessage, local: boolean): void;
     /**
      * Reads blobs needed to load BlobManager from storage.

package/dist/blobManager.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"blobManager.d.ts","sourceRoot":"","sources":["../src/blobManager.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAGH,OAAO,EAAE,YAAY,EAAE,mBAAmB,EAAE,MAAM,iCAAiC,CAAC;AACpF,OAAO,EAAE,uBAAuB,EAAE,MAAM,oCAAoC,CAAC;AAC7E,OAAO,EAEN,yBAAyB,EACzB,aAAa,EACb,MAAM,sCAAsC,CAAC;AAO9C,OAAO,EAKN,iBAAiB,EACjB,MAAM,8BAA8B,CAAC;AACtC,OAAO,EACN,iBAAiB,EACjB,uBAAuB,EACvB,MAAM,+CAA+C,CAAC;AAQvD,OAAO,EACN,sBAAsB,EACtB,qBAAqB,EACrB,iBAAiB,EACjB,MAAM,qCAAqC,CAAC;AAC7C,OAAO,EAAE,gBAAgB,EAA8B,MAAM,oBAAoB,CAAC;AAKlF;;;;;;GAMG;AACH,qBAAa,UAAW,YAAW,YAAY,CAAC,eAAe,CAAC;aAc9C,IAAI,EAAE,MAAM;aACZ,YAAY,EAAE,mBAAmB;IAC1C,GAAG,EAAE,MAAM,OAAO,CAAC,GAAG,CAAC;IAf/B,OAAO,CAAC,QAAQ,CAAkB;IAElC,IAAW,YAAY,IAAI,YAAY,CAEtC;IAED,IAAW,UAAU,IAAI,OAAO,CAE/B;IAED,SAAgB,YAAY,EAAE,MAAM,CAAC;gBAGpB,IAAI,EAAE,MAAM,EACZ,YAAY,EAAE,mBAAmB,EAC1C,GAAG,EAAE,MAAM,OAAO,CAAC,GAAG,CAAC;IAKxB,WAAW;IAIX,IAAI,CAAC,MAAM,EAAE,YAAY;CAGhC;AAmBD;;GAEG;AACH,MAAM,WAAW,oBAAoB;IACpC,GAAG,CAAC,EAAE,MAAM,EAAE,CAAC;IACf,aAAa,CAAC,EAAE,CAAC,MAAM,EAAE,MAAM,CAAC,EAAE,CAAC;CACnC;AAID,oBAAY,mBAAmB,GAAG,IAAI,CACrC,iBAAiB,EACjB,aAAa,GAAG,WAAW,GAAG,QAAQ,GAAG,eAAe,CACxD,GACA,IAAI,CAAC,gBAAgB,EAAE,+BAA+B,CAAC,GACvD,iBAAiB,CAAC,uBAAuB,CAAC,CAAC;AAwB5C,MAAM,WAAW,aAAa;IAC7B,CAAC,EAAE,EAAE,MAAM,GAAG;QAAE,IAAI,EAAE,MAAM,CAAA;KAAE,CAAC;CAC/B;AAED,MAAM,WAAW,kBAAkB;IAClC,CAAC,KAAK,EAAE,gBAAgB,EAAE,QAAQ,EAAE,MAAM,IAAI,OAAE;CAChD;AAED,qBAAa,WAAY,SAAQ,iBAAiB,CAAC,kBAAkB,CAAC;IA8CpE,OAAO,CAAC,QAAQ,CAAC,YAAY;IAE7B,OAAO,CAAC,QAAQ,CAAC,UAAU;IAC3B;;;;;;;;;OASG;IACH,OAAO,CAAC,QAAQ,CAAC,gBAAgB;IAGjC,OAAO,CAAC,QAAQ,CAAC,aAAa;IAG9B,OAAO,CAAC,QAAQ,CAAC,aAAa;IAC9B,OAAO,CAAC,QAAQ,CAAC,OAAO;IAExB,OAAO,CAAC,QAAQ,CAAC,8BAA8B;IAnEhD,gBAAuB,QAAQ,YAAY;IAC3C,OAAO,CAAC,MAAM,CAAC,QAAQ,CAAC,qBAAqB,CAAoB;IACjE,OAAO,CAAC,QAAQ,CAAC,EAAE,CAAoB;IAEvC;;;;;;OAMG;IACH,OAAO,CAAC,QAAQ,CAAC,aAAa,CAAkC;IAEhE;;;;OAIG;IACH,OAAO,CAAC,QAAQ,CAAC,YAAY,CAAuC;IAEpE;;;;OAIG;IACH,OAAO,CAAC,QAAQ,CAAC,WAAW,CAAoC;IAEhE,OAAO,CAAC,QAAQ,CAAC,cAAc,CAO7B;IAEF,6EAA6E;IAC7E,OAAO,CAAC,QAAQ,CAAC,oBAAoB,CAAU;IAC/C;;;OAGG;IACH,OAAO,CAAC,QAAQ,CAAC,eAAe,CAA0B;gBAGxC,YAAY,EAAE,mBAAmB,EAClD,QAAQ,EAAE,oBAAoB,EACb,UAAU,EAAE,MAAM,uBAAuB;IAC1D;;;;;;;;;OASG;IACc,gBAAgB,EAAE,CAAC,OAAO,EAAE,MAAM,EAAE,SAAS,CAAC,EAAE,MAAM,KAAK,IAAI,EAG/D,aAAa,EAAE,CAAC,QAAQ,EAAE,MAAM,KAAK,IAAI,EAGzC,aAAa,EAAE,CAAC,QAAQ,EAAE,MAAM,KAAK,OAAO,EAC5C,OAAO,EAAE,mBAAmB,EAC7C,YAAY,2BAAoB,EACf,8BAA8B,EAAE,MAAM,MAAM,GAAG,SAAS;IAyB1E,OAAO,KAAK,qBAAqB,GAIhC;IAED,IAAW,wBAAwB,IAAI,OAAO,CAE7C;IAED,IAAW,eAAe,IAAI,OAAO,CAKpC;IAED;;OAEG;IACU,WAAW;IAcxB;;;OAGG;IACH,OAAO,CAAC,cAAc;IAUtB;;;OAGG;IACH,OAAO,KAAK,UAAU,GAerB;IAEY,OAAO,CAAC,MAAM,EAAE,MAAM,GAAG,OAAO,CAAC,eAAe,CAAC;IAmC9D,OAAO,CAAC,aAAa;YAUP,kBAAkB;IAUnB,UAAU,CAAC,IAAI,EAAE,eAAe,GAAG,OAAO,CAAC,YAAY,CAAC,eAAe,CAAC,CAAC;YA4BxE,UAAU;IAYxB;;;OAGG;IACH,OAAO,CAAC,cAAc;IAItB,OAAO,CAAC,qBAAqB;IAS7B,OAAO,CAAC,eAAe;YAoDT,cAAc;IAkB5B,OAAO,CAAC,mBAAmB;IAkC3B;;;;OAIG;IACI,QAAQ,CAAC,QAAQ,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,GAAG,SAAS;IAoB7D,OAAO,CAAC,WAAW;IA0BZ,mBAAmB,CAAC,OAAO,EAAE,yBAAyB,EAAE,KAAK,EAAE,OAAO;IA2C7E;;;;;OAKG;WACiB,IAAI,CACvB,SAAS,EAAE,aAAa,GAAG,SAAS,EACpC,YAAY,EAAE,CAAC,EAAE,EAAE,MAAM,KAAK,OAAO,CAAC,CAAC,MAAM,EAAE,MAAM,CAAC,EAAE,CAAC,GACvD,OAAO,CAAC,oBAAoB,CAAC;IAehC;;OAEG;IACH,OAAO,CAAC,IAAI;IAgBL,SAAS,CAAC,gBAAgB,CAAC,EAAE,iBAAiB,GAAG,qBAAqB;IA2B7E;;;;;OAKG;IACI,SAAS,CAAC,MAAM,GAAE,OAAe,GAAG,sBAAsB;IAejE;;;OAGG;IACI,kBAAkB,CAAC,YAAY,EAAE,MAAM,EAAE,GAAG,IAAI;IAIvD;;;;;OAKG;IACI,qBAAqB,CAAC,oBAAoB,EAAE,MAAM,EAAE,GAAG,MAAM,EAAE;IAUtE;;;;;;;;;;OAUG;IACH,OAAO,CAAC,4BAA4B;IAwCpC;;;;OAIG;IACI,sBAAsB,CAAC,gBAAgB,EAAE,MAAM,EAAE;IAsBxD;;;OAGG;IACH,OAAO,CAAC,kBAAkB;IAiDnB,gBAAgB,CAAC,KAAK,EAAE,GAAG,CAAC,MAAM,EAAE,MAAM,CAAC;IAoB3C,eAAe,IAAI,aAAa;CAOvC"}
+{"version":3,"file":"blobManager.d.ts","sourceRoot":"","sources":["../src/blobManager.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAGH,OAAO,EAAE,YAAY,EAAE,mBAAmB,EAAE,MAAM,iCAAiC,CAAC;AACpF,OAAO,EAAE,uBAAuB,EAAE,MAAM,oCAAoC,CAAC;AAC7E,OAAO,EAEN,yBAAyB,EACzB,aAAa,EACb,MAAM,sCAAsC,CAAC;AAO9C,OAAO,EAKN,iBAAiB,EACjB,MAAM,8BAA8B,CAAC;AACtC,OAAO,EACN,iBAAiB,EACjB,uBAAuB,EACvB,MAAM,+CAA+C,CAAC;AACvD,OAAO,EAAe,uBAAuB,EAAE,MAAM,uCAAuC,CAAC;AAO7F,OAAO,EACN,sBAAsB,EACtB,qBAAqB,EACrB,iBAAiB,EACjB,MAAM,qCAAqC,CAAC;AAE7C,OAAO,EAAE,gBAAgB,EAA8B,MAAM,oBAAoB,CAAC;AAKlF;;;;;;GAMG;AACH,qBAAa,UAAW,YAAW,YAAY,CAAC,eAAe,CAAC;aAc9C,IAAI,EAAE,MAAM;aACZ,YAAY,EAAE,mBAAmB;IAC1C,GAAG,EAAE,MAAM,OAAO,CAAC,GAAG,CAAC;IAf/B,OAAO,CAAC,QAAQ,CAAkB;IAElC,IAAW,YAAY,IAAI,YAAY,CAEtC;IAED,IAAW,UAAU,IAAI,OAAO,CAE/B;IAED,SAAgB,YAAY,EAAE,MAAM,CAAC;gBAGpB,IAAI,EAAE,MAAM,EACZ,YAAY,EAAE,mBAAmB,EAC1C,GAAG,EAAE,MAAM,OAAO,CAAC,GAAG,CAAC;IAKxB,WAAW;IAIX,IAAI,CAAC,MAAM,EAAE,YAAY;CAGhC;AAmBD;;GAEG;AACH,MAAM,WAAW,oBAAoB;IACpC,GAAG,CAAC,EAAE,MAAM,EAAE,CAAC;IACf,aAAa,CAAC,EAAE,CAAC,MAAM,EAAE,MAAM,CAAC,EAAE,CAAC;CACnC;AAID,oBAAY,mBAAmB,GAAG,IAAI,CACrC,iBAAiB,EACjB,aAAa,GAAG,WAAW,GAAG,QAAQ,GAAG,eAAe,CACxD,GACA,IAAI,CAAC,gBAAgB,EAAE,+BAA+B,CAAC,GACvD,iBAAiB,CAAC,uBAAuB,CAAC,CAAC;AAuB5C,MAAM,WAAW,aAAa;IAC7B,CAAC,EAAE,EAAE,MAAM,GAAG;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,UAAU,CAAC,EAAE,MAAM,CAAC;QAAC,eAAe,CAAC,EAAE,MAAM,CAAA;KAAE,CAAC;CAC9E;AAED,MAAM,WAAW,kBAAkB;IAClC,CAAC,KAAK,EAAE,gBAAgB,EAAE,QAAQ,EAAE,MAAM,IAAI,OAAE;CAChD;AAED,qBAAa,WAAY,SAAQ,iBAAiB,CAAC,kBAAkB,CAAC;IAgDpE,OAAO,CAAC,QAAQ,CAAC,YAAY;IAE7B,OAAO,CAAC,QAAQ,CAAC,UAAU;IAc3B,OAAO,CAAC,QAAQ,CAAC,aAAa;IAG9B,OAAO,CAAC,QAAQ,CAAC,aAAa;IAC9B,OAAO,CAAC,QAAQ,CAAC,OAAO;IAExB,OAAO,CAAC,QAAQ,CAAC,cAAc;IArEhC,gBAAuB,QAAQ,YAAY;IAC3C,OAAO,CAAC,MAAM,CAAC,QAAQ,CAAC,qBAAqB,CAAoB;IACjE,OAAO,CAAC,QAAQ,CAAC,EAAE,CAAoB;IAEvC;;;;;;OAMG;IACH,OAAO,CAAC,QAAQ,CAAC,aAAa,CAAkC;IAEhE;;;;OAIG;IACH,OAAO,CAAC,QAAQ,CAAC,YAAY,CAAuC;IAEpE;;;;OAIG;IACH,OAAO,CAAC,QAAQ,CAAC,WAAW,CAAoC;IAEhE,OAAO,CAAC,QAAQ,CAAC,cAAc,CAO7B;IAEF,6EAA6E;IAC7E,OAAO,CAAC,QAAQ,CAAC,oBAAoB,CAAU;IAC/C;;;OAGG;IACH,OAAO,CAAC,QAAQ,CAAC,eAAe,CAA0B;IAE1D,OAAO,CAAC,QAAQ,CAAC,gBAAgB,CAAgD;gBAG/D,YAAY,EAAE,mBAAmB,EAClD,QAAQ,EAAE,oBAAoB,EACb,UAAU,EAAE,MAAM,uBAAuB;IAC1D;;;;;;;;;OASG;IACH,gBAAgB,EAAE,CAAC,OAAO,EAAE,MAAM,EAAE,SAAS,CAAC,EAAE,MAAM,KAAK,IAAI,EAG9C,aAAa,EAAE,CAAC,QAAQ,EAAE,MAAM,KAAK,IAAI,EAGzC,aAAa,EAAE,CAAC,QAAQ,EAAE,MAAM,KAAK,OAAO,EAC5C,OAAO,EAAE,mBAAmB,EAC7C,YAAY,2BAAoB,EACf,cAAc,EAAE,CAAC,KAAK,CAAC,EAAE,uBAAuB,KAAK,IAAI;IAuE3E,OAAO,KAAK,qBAAqB,GAIhC;IAED,IAAW,wBAAwB,IAAI,OAAO,CAE7C;IAED,IAAW,eAAe,IAAI,OAAO,CAKpC;IAED;;OAEG;IACU,WAAW;IAcxB;;;OAGG;IACH,OAAO,CAAC,cAAc;IAUtB;;;OAGG;IACH,OAAO,KAAK,UAAU,GAerB;IAEY,OAAO,CAAC,MAAM,EAAE,MAAM,GAAG,OAAO,CAAC,eAAe,CAAC;IAmC9D,OAAO,CAAC,aAAa;YAUP,kBAAkB;IAUnB,UAAU,CAAC,IAAI,EAAE,eAAe,GAAG,OAAO,CAAC,YAAY,CAAC,eAAe,CAAC,CAAC;YA4BxE,UAAU;IAYxB;;;OAGG;IACH,OAAO,CAAC,cAAc;IAItB,OAAO,CAAC,qBAAqB;IAS7B,OAAO,CAAC,eAAe;YAkDT,cAAc;IAkB5B,OAAO,CAAC,mBAAmB;IAiC3B;;;;OAIG;IACI,QAAQ,CAAC,QAAQ,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,GAAG,SAAS;IAkBtD,mBAAmB,CAAC,OAAO,EAAE,yBAAyB,EAAE,KAAK,EAAE,OAAO;IA2C7E;;;;;OAKG;WACiB,IAAI,CACvB,SAAS,EAAE,aAAa,GAAG,SAAS,EACpC,YAAY,EAAE,CAAC,EAAE,EAAE,MAAM,KAAK,OAAO,CAAC,CAAC,MAAM,EAAE,MAAM,CAAC,EAAE,CAAC,GACvD,OAAO,CAAC,oBAAoB,CAAC;IAehC;;OAEG;IACH,OAAO,CAAC,IAAI;IAgBL,SAAS,CAAC,gBAAgB,CAAC,EAAE,iBAAiB,GAAG,qBAAqB;IA2B7E;;;;;OAKG;IACI,SAAS,CAAC,MAAM,GAAE,OAAe,GAAG,sBAAsB;IAejE;;;OAGG;IACI,kBAAkB,CAAC,YAAY,EAAE,MAAM,EAAE,GAAG,IAAI;IAIvD;;;;;OAKG;IACI,qBAAqB,CAAC,oBAAoB,EAAE,MAAM,EAAE,GAAG,MAAM,EAAE;IAUtE;;;;;;;;;;OAUG;IACH,OAAO,CAAC,4BAA4B;IAwCpC;;;;OAIG;IACI,sBAAsB,CAAC,gBAAgB,EAAE,MAAM,EAAE;IAsBxD;;;OAGG;IACH,OAAO,CAAC,kBAAkB;IAiDnB,gBAAgB,CAAC,KAAK,EAAE,GAAG,CAAC,MAAM,EAAE,MAAM,CAAC;IAoB3C,eAAe,IAAI,aAAa;CAavC"}

package/dist/blobManager.js

@@ -10,6 +10,7 @@ const runtime_utils_1 = require("@fluidframework/runtime-utils");
 const common_utils_1 = require("@fluidframework/common-utils");
 const container_definitions_1 = require("@fluidframework/container-definitions");
 const telemetry_utils_1 = require("@fluidframework/telemetry-utils");
+const container_utils_1 = require("@fluidframework/container-utils");
 const containerRuntime_1 = require("./containerRuntime");
 const gc_1 = require("./gc");
 const throttler_1 = require("./throttler");
@@ -86,15 +87,14 @@ class BlobManager extends common_utils_1.TypedEventEmitter {
     blobRequested, 
     // Called to check if a blob has been deleted by GC.
     // blobPath's format - `/<BlobManager.basePath>/<blobId>`.
-    isBlobDeleted, runtime, stashedBlobs = {}, getCurrentReferenceTimestampMs) {
+    isBlobDeleted, runtime, stashedBlobs = {}, closeContainer) {
         super();
         this.routeContext = routeContext;
         this.getStorage = getStorage;
-        this.sendBlobAttachOp = sendBlobAttachOp;
         this.blobRequested = blobRequested;
         this.isBlobDeleted = isBlobDeleted;
         this.runtime = runtime;
-        this.getCurrentReferenceTimestampMs = getCurrentReferenceTimestampMs;
+        this.closeContainer = closeContainer;
         /**
          * Blobs which have not been uploaded or for which we have not yet seen a BlobAttach op round-trip.
          * Until we see the op round-trip, there is a possibility we may need to re-upload the blob, so
@@ -127,6 +127,21 @@ class BlobManager extends common_utils_1.TypedEventEmitter {
         // Begin uploading stashed blobs from previous container instance
         Object.entries(stashedBlobs).forEach(([localId, entry]) => {
             const blob = (0, common_utils_1.stringToBuffer)(entry.blob, "base64");
+            if (entry.minTTLInSeconds && entry.uploadTime) {
+                const timeLapseSinceLocalUpload = (Date.now() - entry.uploadTime) / 1000;
+                // stashed entries with more than half-life in storage will not be reuploaded
+                if (entry.minTTLInSeconds - timeLapseSinceLocalUpload > entry.minTTLInSeconds / 2) {
+                    this.pendingBlobs.set(localId, {
+                        blob,
+                        status: PendingBlobStatus.OfflinePendingOp,
+                        handleP: new common_utils_1.Deferred(),
+                        uploadP: undefined,
+                        uploadTime: entry.uploadTime,
+                        minTTLInSeconds: entry.minTTLInSeconds,
+                    });
+                    return;
+                }
+            }
             this.pendingBlobs.set(localId, {
                 blob,
                 status: PendingBlobStatus.OfflinePendingUpload,
@@ -134,6 +149,30 @@ class BlobManager extends common_utils_1.TypedEventEmitter {
                 uploadP: this.uploadBlob(localId, blob),
             });
         });
+        this.sendBlobAttachOp = (localId, blobId) => {
+            const pendingEntry = this.pendingBlobs.get(localId);
+            if ((pendingEntry === null || pendingEntry === void 0 ? void 0 : pendingEntry.uploadTime) && (pendingEntry === null || pendingEntry === void 0 ? void 0 : pendingEntry.minTTLInSeconds)) {
+                const secondsSinceUpload = (Date.now() - pendingEntry.uploadTime) / 1000;
+                const expired = pendingEntry.minTTLInSeconds - secondsSinceUpload < 0;
+                this.mc.logger.sendTelemetryEvent({
+                    eventName: "sendBlobAttach",
+                    entryStatus: pendingEntry.status,
+                    secondsSinceUpload,
+                    minTTLInSeconds: pendingEntry.minTTLInSeconds,
+                    expired,
+                });
+                if (expired) {
+                    // we want to avoid submitting ops with broken handles
+                    this.closeContainer(new container_utils_1.GenericError("Trying to submit a BlobAttach for expired blob", undefined, {
+                        localId,
+                        blobId,
+                        entryStatus: pendingEntry.status,
+                        secondsSinceUpload,
+                    }));
+                }
+            }
+            return sendBlobAttachOp(localId, blobId);
+        };
     }
     get pendingOfflineUploads() {
         return Array.from(this.pendingBlobs.values()).filter((e) => e.status === PendingBlobStatus.OfflinePendingUpload);
@@ -266,9 +305,8 @@ class BlobManager extends common_utils_1.TypedEventEmitter {
         (0, common_utils_1.assert)((entry === null || entry === void 0 ? void 0 : entry.status) === PendingBlobStatus.OnlinePendingUpload ||
             (entry === null || entry === void 0 ? void 0 : entry.status) === PendingBlobStatus.OfflinePendingUpload, 0x386 /* Must have pending blob entry for uploaded blob */);
         entry.storageId = response.id;
-        entry.localUploadTime = Date.now();
+        entry.uploadTime = Date.now();
         entry.minTTLInSeconds = response.minTTLInSeconds;
-        entry.serverUploadTime = this.getCurrentReferenceTimestampMs();
         if (this.runtime.connected) {
             if (entry.status === PendingBlobStatus.OnlinePendingUpload) {
                 // Send a blob attach op. This serves two purposes:
@@ -276,7 +314,6 @@ class BlobManager extends common_utils_1.TypedEventEmitter {
                 //    until its storage ID is added to the next summary.
                 // 2. It will create a local ID to storage ID mapping in all clients which is needed to retrieve the
                 //    blob from the server via the storage ID.
-                this.logTimeInfo(entry, "sendBlobAttachResolveTTL");
                 this.sendBlobAttachOp(localId, response.id);
                 if (this.storageIds.has(response.id)) {
                     // The blob is de-duped. Set up a local ID to storage ID mapping and return the blob. Since this is
@@ -340,7 +377,6 @@ class BlobManager extends common_utils_1.TypedEventEmitter {
          * is called on reconnection.
          */
         if (entry.status !== PendingBlobStatus.OnlinePendingOp) {
-            this.logTimeInfo(entry, "sendBlobAttachTransitionOfflineTTL");
             this.sendBlobAttachOp(localId, entry.storageId);
         }
         entry.status =
@@ -359,9 +395,6 @@ class BlobManager extends common_utils_1.TypedEventEmitter {
         const { localId, blobId } = metadata;
         (0, common_utils_1.assert)(localId !== undefined, 0x50d /* local ID not available on reSubmit */);
         const pendingEntry = this.pendingBlobs.get(localId);
-        if (pendingEntry) {
-            this.logTimeInfo(pendingEntry, "sendBlobAttachResubmitTTL");
-        }
         if (!blobId) {
             // We submitted this op while offline. The blob should have been uploaded by now.
             (0, common_utils_1.assert)((pendingEntry === null || pendingEntry === void 0 ? void 0 : pendingEntry.status) === PendingBlobStatus.OfflinePendingOp &&
@@ -370,32 +403,6 @@ class BlobManager extends common_utils_1.TypedEventEmitter {
         }
         return this.sendBlobAttachOp(localId, blobId);
     }
-    logTimeInfo(pendingEntry, eventName) {
-        var _a, _b;
-        let timeLapseSinceLocalUpload = 0;
-        let timeLapseSinceServerUpload = 0;
-        let expiredUsingLocalTime;
-        let expiredUsingServerTime;
-        if (pendingEntry.localUploadTime) {
-            timeLapseSinceLocalUpload = (Date.now() - pendingEntry.localUploadTime) / 1000;
-            expiredUsingLocalTime =
-                ((_a = pendingEntry.minTTLInSeconds) !== null && _a !== void 0 ? _a : 0) - timeLapseSinceLocalUpload < 0 ? true : false;
-        }
-        if (pendingEntry.serverUploadTime) {
-            timeLapseSinceServerUpload = (Date.now() - pendingEntry.serverUploadTime) / 1000;
-            expiredUsingServerTime =
-                ((_b = pendingEntry.minTTLInSeconds) !== null && _b !== void 0 ? _b : 0) - timeLapseSinceServerUpload < 0 ? true : false;
-        }
-        this.mc.logger.sendTelemetryEvent({
-            eventName,
-            entryStatus: pendingEntry.status,
-            timeLapseSinceLocalUpload,
-            timeLapseSinceServerUpload,
-            minTTLInSeconds: pendingEntry.minTTLInSeconds,
-            expiredUsingLocalTime,
-            expiredUsingServerTime,
-        });
-    }
     processBlobAttachOp(message, local) {
         var _a, _b;
         const localId = (_a = message.metadata) === null || _a === void 0 ? void 0 : _a.localId;
@@ -651,7 +658,13 @@ class BlobManager extends common_utils_1.TypedEventEmitter {
     getPendingBlobs() {
         const blobs = {};
         for (const [key, entry] of this.pendingBlobs) {
-            blobs[key] = { blob: (0, common_utils_1.bufferToString)(entry.blob, "base64") };
+            blobs[key] = entry.minTTLInSeconds
+                ? {
+                    blob: (0, common_utils_1.bufferToString)(entry.blob, "base64"),
+                    uploadTime: entry.uploadTime,
+                    minTTLInSeconds: entry.minTTLInSeconds,
+                }
+                : { blob: (0, common_utils_1.bufferToString)(entry.blob, "base64") };
         }
         return blobs;
     }